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EXECUTIVE  SUMMARY 


This  research  demonstrated  that  simulation  could  provide  valuable  information  for 
decision-making  at  the  unit  level  and  provide  much  needed  assistance  in  the  generation  and 
execution  of  a  weekly  flying  schedule.  In  previous  research  a  simulation  model  detailing  an  Air 
Force  Multi-Indenture  Multi-Echelon  (MIME)  repairable  parts  system  was  developed.  That 
simulation  model  was  used  to  explore  the  effects  of  using  commercial  shipping  practices  on  the 
supply  chain.  This  research  expanded  the  simulation  model  developed  for  the  commercial 
transportation  project  to  detail  the  sortie  generation  process.  Using  this  simulation  the  effective 
risk  inherent  in  a  given  weekly  schedule  was  evaluated.  The  concept  was  to  allow  until  level 
logistics  planners  to  input  a  weekly  schedule  and  to  evaluate  that  schedule  based  on  outputs 
received  from  the  simulation.  A  model  of  this  type  also  allows  the  unit  level  logistics  planners  to 
compare  alternative  schedules  and  perform  what-if  analysis. 

There  are  significant  challenges  in  developing  a  tool  that  is  useful  at  the  unit  level.  First, 
the  user  interface  must  be  such  that  the  planners  can  quickly  and  easily  input  the  needed  data.  If 
the  user  interface  is  too  complex  or  confusing  no  value  will  be  added  to  the  decision  process. 
Second,  the  simulation  must  be  detailed  enough  to  provide  useful  outputs  to  the  user,  but  also 
simple  enough  that  the  simulation  time  and  data  requirements  are  minimal. 

This  research  began  by  reviewing  the  relevant  literature,  systems,  and  processes.  Part  of 
the  goal  was  to  illustrate  how  simulation  could  be  used  in  this  context.  Thus,  military  logistics, 
MIME  systems,  and  user  interface  systems  required  review.  Likewise,  the  project  provided  a 
detailed  description  of  the  simulation  model  along  with  the  user  interface  system  and  a 
description  of  data  handling  methods.  In  addition,  a  test  scenario  was  outlined  along  with  detail 
on  how  the  model  handles  this  scenario  from  user  input  to  interpreting  the  output  data. 


in 


r 


Examples  of  what-if  analysis  that  can  be  performed  using  the  same  test  scenario  were  also 
developed.  Through  the  execution  of  this  test  scenario,  the  research  demonstrated  how  this  type 
of  simulation  technology  could  be  useful  in  making  decisions  at  the  unit  level.  The  outputs  will 
provide  the  user  valuable  assistance  in  making  timely  decisions  that  can  mitigate  the  risk 
associated  with  executing  a  flying  schedule. 
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Introduction 


In  this  research  our  goal  is  to  demonstrate  that  simulation  can  provide  valuable 
information  for  decision-making  at  the  unit  level.  In  previous  research,  we  developed  a 
simulation  model  detailing  an  Air  Force  Multi-Indenture  Multi-Echelon  (MIME)  repairable  parts 
system.  We  used  this  simulation  model  to  explore  the  effects  of  using  commercial  shipping 
practices  on  the  supply  chain.  We  expand  the  simulation  model  developed  for  the  commercial 
transportation  project  to  detail  the  sortie  generation  process.  Using  this  simulation,  we  evaluate 
the  effective  risk  inherent  in  a  given  weekly  schedule.  The  concept  is  to  give  until  level  logistics 
planners  the  ability  to  input  a  weekly  schedule  and  to  evaluate  that  schedule  based  on  outputs 
received  from  the  simulation.  A  model  of  this  type  also  allows  the  unit  level  logistics  planners  to 
compare  alternative  schedules  and  perform  what-if  analysis.  Due  to  the  fact  that  much  of  the 
model  has  remained  the  same,  we  have  omitted  certain  model  details.  Please  refer  to  our 
previous  report  if  more  explanation  is  desired.  Our  goal  for  this  report  is  to  concentrate  on  the 
new  material  we  have  developed. 

There  are  a  number  of  challenges  in  developing  a  tool  that  is  usefiil  at  the  unit  level. 
First,  the  user  interface  must  be  such  that  the  planners  can  quickly  and  easily  input  the  needed 
data.  If  the  user  interface  is  too  complex  or  confusing  no  value  will  be  added  to  the  decision 
process.  Second,  the  simulation  must  be  detailed  enough  to  provide  useful  outputs  to  the  user, 
but  also  simplistic  enough  that  the  simulation  time  and  data  requirements  are  minimal. 

In  this  report,  we  begin  by  detailing  the  relevant  literature,  systems,  and  processes.  Part 
of  our  goal  is  to  illustrate  how  simulation  can  be  used  in  this  context.  Thus,  we  must  discuss 
military  logistics,  MIME  systems,  and  user  interface  systems.  The  next  section  of  this  report 
details  our  modeling  approaches.  In  this  section,  we  provide  a  detailed  description  of  the 
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simulation  model  along  with  the  user  interface  system.  We  also  included  in  this  discussion  a 
description  of  our  data  handling  methods.  Finally,  we  outline  a  test  scenario,  and  detail  how  the 
model  handles  this  scenario  from  user  input  to  interpreting  the  output  data.  We  also  show 
example  what-if  analysis  that  can  be  performed  using  the  same  test  scenario.  Through  the 
execution  of  this  test  scenario,  we  demonstrate  how  this  type  of  simulation  technology  can  be 
useful  in  making  decisions  at  the  unit  level.  The  outputs  provided  to  the  user  will  assist  them  in 
making  timely  decisions  that  can  mitigate  the  risk  associated  with  executing  a  flight  schedule. 

2  Sortie  Generation  and  the  MIME  Repairable  Inventory  System 
Our  previous  report  concentrated  on  the  logistical  aspects  of  the  Multi-Indenture  Multi-Echelon 
(MIME)  repairable  parts  inventory  system  used  by  the  United  States  Air  Force.  In  our  previous 
research  we  modeled  the  MIME  supply  chain  in  detail.  In  this  research  we  look  at  the  sortie 
generation  process  that  is  supported  by  the  MIME  supply  chain.  In  the  following  sections,  we 
will  give  a  brief  description  of  an  MIME  repairable  parts  system. 

2.1  MIME  Repairable  Parts  System 

In  this  research  we  will  use  the  previously  developed  MIME  supply  chain  model.  Multi-echelon 
refers  to  the  fact  that  inventory  is  kept  and  repaired  at  multiple  levels  in  the  supply  chain  (i.e. 
depot,  bases,  and  weapons).  This  structure  can  be  seen  in  Figure  1 .  At  each  echelon  actions  are 
performed.  These  actions  have  an  increasing  level  of  sophistication  as  a  part  progresses  up 
through  the  levels.  Sometimes  the  echelons  are  simply  distinguished  by  their  unique  geographic 
location  in  relation  to  the  previous  level.  The  USAF  has  a  two-level  maintenance  system,  where 
the  first  echelon  is  maintenance  performed  on  base  and  a  second  echelon  where  maintenance  is 
performed  at  more  specialized  and  sophisticated  depot. 
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Figure  1:  Multi-echelon  system 

Figure  1  depicts  a  depot  that  supports  three  bases  (Basei,  Base2,  Bases),  each  of  which 
supports  three  weapon  systems  (Weapon^  Weapon  2,  Weapon  3).  This  system  can  be  expanded, 
in  theory,  with  additional  bases,  weapons,  and  depots. 

Parts  moving  through  the  system  are  composed  of  components  and  those  components  are 
in  turn  made  up  of  sub-components.  The  parts  are  said  to  be  multi-indentured  because  of  this 
stratification  of  components.  In  this  context,  a  first-indenture  component  that  is  removed  from 
an  aircraft  is  called  a  line-replaceable-unit  (LRU)  because  this  activity  takes  place  on  the  flight 
line.  LRUs  are  usually  repairable  or  recoverable  and  they  tend  to  be  expensive,  with  low 
demand  at  any  particular  base.  When  a  first-indenture  item  is  taken  apart  in  a  maintenance  shop, 
second-indenture  items  are  replaced;  these  are  called  shop-replaceable  units  (SRUs).  When  an 
item  fails,  it  takes  time  and  expertise  to  diagnose  and  replace  the  SRUs  that  are  responsible.  It 
may  take  specialized  test  equipment,  and  it  may  require  sending  the  item  to  the  next-higher 
echelon  or  depot.  This  extra  time  translates  into  aircraft  downtime.  For  example,  a  fighter 
maintenance  squadron  will  spend  up  to  13  hours  of  maintenance  for  every  one  hour  of  flight 
time. 
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For  this  research,  a  weapon  is  conceptualized  as  an  aircraft.  Each  weapon  is  made  up  of 
multiple  components  called  line  replaceable  units  (LRU)  shown  in  Figure  3.  LRUs  are,  in  turn, 
made  up  of  multiple  sub-components  called  shop  replaceable  units  (SRU).  In  Figure  2,  the 
subscript  i  denotes  the  LRU  for  which  the  SRU  is  a  subcomponent,  while  j  denotes  the  SRU 
type. 


Figure  2:  Hierarchy  of  Weapon  System 


2.2  Failures 

A  weapon  incurs  downtime  when  one  of  its  LRUs  fails.  An  LRU  fails  when  one  or  more  of  its 
component  SRUs  fails.  When  a  failure  occurs,  the  failed  LRU  is  removed  from  the  aircraft  and 
sent  to  repair.  Each  base  has  a  limited  repair  facility  for  failed  parts  and  a  warehouse  to  store  a 
limited  amount  of  inventory.  The  typical  failure  cycle  of  a  simple,  single  echelon  base  is  shown 
in  Figure  3.  Note  that  every  spare  part  shown  is  not  necessarily  the  same  part  that  originally 
failed.  The  representation  of  the  failure  cycle  and  repair  process  becomes  more  complicated  than 
that  depicted  in  Figure  3  when  we  move  to  a  MIME  system. 
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Figure  3:  Failure  Cycle  and  Repair  Process 

2. 2.1  LRU  repair  at  the  Base  level 

As  described  above,  the  failed  LRU  is  removed  from  the  weapon  and  sent  to  an  intermediate 
repair  shop.  At  test  stations,  technicians  perform  a  battery  of  tests  to  determine  the  problem. 
Sometimes,  the  LRU  can  be  fixed  by  straightening  pins,  cleaning  connections,  replacing  fuses, 
re-soldering  connections,  or  re-calibrating.  But  often,  tests  reveal  that  repair  requires  the 
replacement  of  a  failed  SRU.  Once  the  defective  SRU  is  determined,  two  things  can  happen: 

1.  If  a  spare  SRU  is  available  in  the  base’s  inventory,  the  defective  SRU  is  swapped 
with  the  spare.  The  defective  SRU  is  either  repaired  at  the  base  level  and  put  back 
into  inventory,  or  is  sent  to  the  depot  for  repair.  If  the  SRU  is  sent  to  the  depot  for 
repair,  an  order  is  generated  for  the  part. 

2.  If  a  spare  SRU  is  unavailable  in  base  inventory,  the  defective  SRU  is  sent  to  the  depot 
for  repair  and  an  order  is  generated  for  this  part.  The  order  is  marked  to  indicate  that 
the  aircraft  from  which  the  defective  SRU  was  removed  is  non-mission  capable 
awaiting  parts  (MICAP).  This  designation  indicates  that  express-air  shipping  will  be 
used. 
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In  both  scenarios,  base-level  repair  is  characterized  by  swapping  the  failed  SRU  with  a 
serviceable  SRU,  called  filling  the  LRU  hole.  Also,  in  either  case,  if  a  defective  SRU  is  sent  to 
the  depot  for  repair,  an  order  corresponding  to  that  SRU  type  is  sent  to  the  depot.  The  order  is 
filled  from  depot  inventory  and  shipped  back  to  the  base  to  replenish  its  inventory. 

2. 2.2  SRU  repair  at  the  Depot  level 

If  a  base  cannot  repair  a  failed  SRU,  the  base  sends  it  to  the  depot  for  repair.  Upon  receipt  of  a 
SRU  from  a  base,  the  depot  will  send  the  base  a  serviceable  SRU.  The  SRU  received  from  the 
base  will  be  inducted  into  the  depot’s  repair  process  and  upon  repair  will  be  retained  in  the 
depot’s  inventory 

The  depot  tries  to  maintain  a  constant  level  of  inventory  for  itself  and  the  bases,  in 
keeping  with  the  aforementioned  base  stock  inventory  policy.  From  the  depot’s  standpoint, 
whenever  a  defective  part  is  received,  a  serviceable  part  is  shipped.  Similarly,  from  the  base’s 
standpoint,  whenever  a  defective  part  is  shipped,  a  serviceable  part  is  received. 

2.3  Transportation 

The  MIME  inventory  and  repair  system  must  also  account  for  the  transportation  of  parts  between 
depots  and  bases.  In  this  research  we  will  only  model  the  two  traditional  ways  for  shipping  parts 
in  an  Air  Force  supply  chain:  less-than-truckload  (LTL)  and  MICAP  (expedited  shipping).  LTL 
shipments  of  parts  are  made  on  a  daily  bases  between  the  depot  and  the  bases,  and  MICAP 
shipments  are  made  when  the  shipment  must  be  expedited  according  to  preordained 
specifications.  For  a  more  comprehensive  explanation  of  shipping  practices  please  refer  to  our 
previous  research  report. 
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2.4  Previous  MIME  Models 

The  MIME  system  has  been  the  focus  of  much  research  over  the  years.  A  review  of  the 

literature  shows  that  most  of  this  research  has  been  devoted  to  the  development  of  mathematical 

models  that  explore  redistribution  of  resources,  suggest  the  optimized  allocation  of  stock  and 

offer  a  means  of  system  evaluation.  These  models  trace  their  roots  back  to  a  paper  by 

Sherbrooke  (1968)  in  which  he  introduces  METRIC  (Multi-Echelon-Technique-for- 

Recoverable-Item-Control).  The  basic  model  tries  to  optimize  stock  level  of  items  at  a  given 

echelon.  It  is  depicted  in  Equation  1  “Basic  Stock  Level”  (Adapted  from  Sherbrooke,  2004). 

Stock  Level  =  (number  of  units  on  hand)  +  (the  number  of  units  due  in  from  repair  or  (1) 
re-supply)  -  (the  number  of  units  on  backorder) 

See  for  example  the  models  developed  by  Muckstandt  (1973),  Nahmias  (2001)  Slay  (1984),  Slay 
et  al,  (1996),  Graves  (1985)  and  Sherbrooke  (1986)  or  for  a  general  overview  of  all  models  see 
Kennedy,  et  al  (2002).  METRIC  is  not  a  true  MIME  model;  it  focuses  in  on  the  multi-echelon 
nature  of  the  supply  chain  not  the  multi-indentured  characteristic  of  the  components.  The  model 
does  lay  the  cornerstone  assumptions  for  the  later  models.  Some  of  these  assumptions  include: 

•  There  is  no  lateral  supply  between  bases 

•  There  are  no  condemnations  (all  failed  components  are  repairable) 

•  The  level  (base  or  depot)  at  which  the  repair  is  performed  depends  only  on  the 
complexity  of  the  repair 

•  There  is  no  waiting  or  batching  of  components  before  a  repair  is  initiated 

These  limiting  assumptions  hamper  their  ability  to  provide  detailed  insight  into  the  workings  of 
the  system.  METRIC  and  similar  models  are  generally  concentrated  on  the  mathematical 
optimization  of  the  inventory  costs  and  service  levels  associated  with  a  potential  spares  inventory 
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policy,  safety  stocks  and  so  on.  Unfortunately,  most  of  these  methodologies  are  too  complex, 
abstract  or  oversimplified,  thus  reducing  their  usefulness  for  a  maintenance  manager  (Braglia  et 
al,  2004)  like  the  Production  Supervisor. 

The  literature  regarding  the  MIME  inventory  system  (namely  the  models  built  on 
Sherbrooke’s  METRIC  (1968)  and  Muckstadt’s  model  METRIC  (1973))  makes  a  number  of 
simplifying  assumptions.  One  of  these  assumptions  relaxed  in  our  model  is: 

■  SRU  and  LRU  Failures.  Both  Sherbrooke  (1968)  and  Muckstadt  (1973)  assume  that 
a  failed  weapon  is  the  result  of  only  one  failed  LRU  and  that  a  failed  LRU  is  the  result 

of  only  one  failed  SRU.  However,  since  parts  can  often  fail  in  clusters,  as  the  failure 

of  an  LRU  can  cause  the  failure  of  another  LRU,  it  is  more  appropriate  to  disregard 
this  assumption.  In  our  model  we  allow  multiple  LRUs  to  fail. 

3  Literature  Review  of  Sortie  Generation 

The  sortie  generation  process  consists  of  two  main  parts:  planning  and  execution.  The  first  part 
involves  a  sortie  plan  that  is  revised  at  the  yearly,  quarterly,  monthly,  and  weekly  basis  where  it 
emerges  as  an  executable  schedule  of  events.  In  the  following  sections  we  will  review  both  of 
these  two  main  parts  to  the  sortie  generation  process.  A  large  portion  of  the  information  below 
was  taken  from  United  States  Air  Force  Instruction  Document  AFI21-101  (2002). 

3.1  Planning 

The  planning  process  is  an  iterative  negotiation  process  in  which  operations  informs  maintenance 
about  their  requirements,  and  maintenance  reviews  their  capacity  and  informs  operations  about 
what  they  can  provide.  This  process  happens  at  many  different  levels  of  the  command  chain 
over  the  period  of  a  year  finally  resulting  in  executable  weekly  schedules. 
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3.1.1  Annual  Plan 


The  annual  plan  is  a  coordinated  effort  between  operations  and  maintenance.  The  goal  of  this 
plan  is  to  “evaluate  the  capability  of  maintenance  to  support  the  annual  flying-hour  program”. 
Both  the  number  and  length  of  sorties  along  with  the  maintenance  requirements  are  examined  in 
attempting  to  devise  a  plan  that  will  meet  the  UTE  rate  standards/goals. 

Provided  in  this  plan  are: 

•  Airframe,  personnel,  and  facility  capabilities 

•  Required  flying  hours,  estimated  sorties,  and  missions  provided  in  monthly  increments 

•  Flying  days  by  month 

•  Aircraft/Aircrew  requirements 

•  Known  and  projected  Temporary  Duty  and  Special  Missions  Requirements 

•  Configuration  and  Munitions  Requirements 

•  Programmed  Depot  Maintenance,  depot,  and  transfer  schedule 

•  Established  number  of  available  by  month 

•  Estimated  monthly  attrition  factor 

•  Recommended  block  scheduling  pattern  to  meet  requirements 

•  Statement  of  limitations 
3.1.2  Quarterly  Plan 

The  quarterly  plan  addresses  the  ability  of  maintenance  to  meet  the  established  plan  for  the 
quarter  laid  out  in  the  annual  plan.  Scheduling  is  called  upon  to  ensure  that  the  quarterly  plans 
are  as  detailed  as  possible.  It  is  important  to  note  that  a  rolling  3-month  plan  briefed  each  month 
at  the  monthly  meeting  will  suffice  to  fulfill  the  quarterly  planning  requirement.  The  intent  of 
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the  quarterly  plan  is  to  further  develop  the  long-range  plan  initiated  in  the  annual  planning 
session. 

3.1.3  Monthly  Schedule 

The  monthly  plan  is  intended  to  add  further  detail  to  the  quarterly  plan.  The  monthly  plan  is 
required  to  “forecast  and  monitor  requirements  for  the  current  month  and  next  two  months.”  The 
monthly  schedule  will  detail  the  total  number  of  sorties,  attrition  sorties  added,  and  the  number 
of  sorties  for  each  unit.  This  monthly  plan  is  developed  over  the  course  of  the  month  proceeding 
the  month  being  scheduled. 

Specifically: 

•  Week  1  -  Estimated  operational  needs  for  the  following  month  are  provided  in  as  much 
detail  as  possible.  This  is  meant  to  include  times  for  takeoff,  landing,  and  flying 
windows. 

•  Week  2  -  Maintenance  tells  operations  whether  the  requirements  for  the  next  month  can 
be  met  along  with  the  limitations  that  may  prevent  the  fulfillment  of  the  requirements. 
Any  needed  adjustments  are  made  to  bring  agreement  between  operations  and 
maintenance  requirements. 

•  Week  3  -  At  this  point  operations  and  maintenance  formalize  the  next  month’s  schedule 
prior  to  presenting  it  to  the  wing  (WG)  commander. 

•  Week  4  -  Upon  the  WG  commander’s  approval,  the  monthly  schedule  is  published  and 
distributed  no  later  than  5  days  prior  to  the  beginning  of  the  planned  month. 

Included  in  the  monthly  schedule  are: 


10 


•  Detailed  utilization  calendar  specifying  total  aircraft  flying  hours,  total  sorties,  missions, 
alert  requirements,  scheduled  sortie  and  mission  requirements,  daily  turn  plans  for  each 
mission  design  series. 

•  Maintenance  workload  requirements 

•  All  scheduled  aircraft  inspections 

•  Total  ordinance  requirements 

•  TRAP  requirements 

•  All  support  requirements  i.e.  supply  requirements,  food  service  requirements,  etc. . . 

3.1.4  Weekly  Schedule 

“Weekly  scheduling  is  the  final  refinement  of  the  monthly  plan  and  results  in  the  weekly  flying 
and  maintenance  schedule.”  The  weekly  schedule  is  distributed  no  later  than  1200  Friday 
morning  before  the  effective  week  and  includes: 

•  Aircraft  takeoff  and  landing  times  including  aircraft  tail  numbers  (primary  and  spares) 

•  Sortie  sequence  numbers 

•  Configuration  requirements 

•  Munitions  requirements 

•  Fuel  loads 

•  Special  or  particular  mission  support  requirements 

•  Alert  requirements 

•  Exercise  vulnerability 

•  Deployments 

•  Off  base  sorties 

•  Equipment  training  requirements 
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3.1.5  Changes  to  the  Weekly  Schedule 

In  executing  the  weekly  schedule  there  are  many  things  that  could  happen  to  result  in  further 
planning  or  manipulation  of  the  schedule.  There  are  three  types  of  changed  which  can  be  made 
to  the  weekly  schedule  after  its  distribution: 

•  Pen-and-ink  are  intended  to  allow  for  minor  changes  to  the  weekly  schedule  which  arise 
due  to  fluctuation  in  aircraft  availability.  Allowable  changes  include  tail  numbers, 
takeoff/landing  times,  etc... 

•  Interchanges  or  swapping  tail  numbers  are  intended  to  prevent  unnecessary 
reconfigurations  and  expenditure  of  work  hours. 

•  Configuration  changes  in  the  required  configuration  of  units  can  be  made  to  reduce  man 
hours  as  long  as  the  requirements  of  the  sortie  can  be  met. 

3.1.6  Flying  Schedule  Deviations 

The  recording  of  schedule  deviations  is  only  applied  to  the  printed  weekly  schedule.  We  refer 
the  reader  to  Air  Combat  Command  (ACC)  Instruction  21-165  for  a  complete  listing  of  deviation 
scenarios.  These  deviations  directly  impact  the  measures  of  scheduling  effectiveness.  There  are 
multiple  types  of  schedule  deviations  that  are  grouped  into  two  categories  as  follows. 

Ground  Deviations 

•  Addition  -  The  addition  of  an  aircraft/sortie  to  the  schedule  not  previously  printed  on  the 
weekly  schedule. 

•  Cancellation  -  An  aircraft  that  is  removed  from  the  printed  schedule  for  any  reason. 

•  Early  Takeoff  -  A  scheduled  sortie  launching  more  than  30  minutes  prior  to  the 
scheduled  takeoff  time. 
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•  Ground  Abort  -  An  event  preventing  a  “crew  ready”  aircraft  from  becoming  airborne.  A 
ground  abort  by  itself  is  not  a  deviation,  but  can  cause  a  deviation  in  the  form  of  a 
cancellation  of  late  takeoff. 

•  Late  Takeoff  -  A  scheduled  sortie  launching  more  than  15  minutes  after  the  scheduled 
takeoff  time. 

•  Spare  -  A  spare  aircraft  launched  instead  of  the  scheduled  aircraft. 

•  Interchange  -  Tail  number  swaps  can  be  made  up  until  the  crew  ready  time. 

Air  Deviations 

•  Air  Abort  -  A  sortie  which  cannot  be  completed  after  takeoff  for  any  reason.  Air  aborts 
are  considered  a  sortie  flown  when  reporting  total  sorties. 

•  Air  Abort,  IFE  -  An  air  abort  resulting  in  a  in-flight  emergency 

•  Early  Landing  -  A  sortie  landing  more  than  15  min  before  the  scheduled  landing  time 
(not  used  when  computing  FSE). 

•  IFE  -  A  situation  resulting  in  an  in-flight  emergency  after  the  mission  has  been 
accomplished. 

•  Late  Landing  -  A  sortie  landing  more  than  15  min  after  the  scheduled  landing  time  (not 
used  when  computing  FSE). 

3. 1. 7  Flying  Schedule  Deviation  Causes 

Each  deviation  from  the  weekly  schedule  is  required  to  be  assigned  a  primary  cause.  The 

primary  cause  categories  of  interest  are  as  follows: 

•  Maintenance  -  Deviations  resulting  from  unscheduled  maintenance. 

•  Supply  -  Deviations  resulting  from  a  Partially  Mission  Capable  or  Not  Mission  Capable 
condition  due  to  lack  of  supply. 
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•  Weather  -  Deviations  due  to  severe  weather. 

•  Sympathy  -  Deviations  resulting  from  aircraft  deviations  within  a  scheduled  group  of 
aircraft. 

3.1.8  Schedule  Exceptions 

Exceptions  are  defined  as  allowed  schedule  deviations.  The  situations  in  which  schedule 
exceptions  are  as  follows: 

•  Adverse  Weather  -  Sorties  can  be  added  to  make  up  for  cancelled  sorties  due  to  severe 
weather.  This  number  cannot  exceed  the  difference  between  the  actual  sorties  canceled 
and  the  planned  attrition  number. 

•  Achievement  of  Utilization  Rate  -  The  operations  group  commander  is  encouraged  to 
modify  or  cancel  all  or  part  of  the  schedule  when  they  are  reasonably  assured  the  UTE 
rate  goal  for  the  month  will  be  met. 

3.1.9  Flying  Schedule  Effectiveness 

Equation  2  details  the  calculation  for  Scheduling  Effectiveness.  Deviations  have  a  major  impact 
on  this  metric  and  are  avoided  by  using  the  ability  to  change  the  weekly  schedule  while  it  is 
being  executed.  Changing  the  schedule  while  it  is  being  executed  is  complicated  due  to  the  high 
number  of  factors  at  play  and  the  far  reaching  consequences  for  the  unit  based  maintenance 
planners. 

Total  Sorties  Scheduled  =  Home  Base  Schedule 

Total  Sorties  Scheduled  -  Total  Deviations  (2) 

Scheduling  Effectiveness  = - *  100 

Total  Sorties  Scheduled 
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3.2  Execution 


The  sortie  generation  process  is  the  cycle  of  inspection,  service,  flight  and  maintenance  used  to 
maintain  a  viable  Air  Force  Wing.  The  requirements  for  maintenance  and  the  sortie  generation 
process  are  found  in  AFI21-101  (2002).  It  is  described  generally  in  Figure  4. 


Figure  4:  Sortie  Generation  Process  (Adapted  from  Faas,  2003) 

Typically  the  sortie  generation  cycle  begins  with  the  recovery  of  an  aircraft  from  another 
mission.  After  proceeding  to  the  parking  location  and  shutdown  of  the  engine,  post-flight 
servicing  is  conducted.  During  this  time  the  aircrew  debriefs  the  maintenance  crew  on  any 
discrepancies  which  are  then  discussed,  documented,  and  recorded  in  a  computerized 
information  system,  the  Core  Automated  Maintenance  System  (CAMS).  Aircraft  and 
equipment  readiness  is  the  maintenance  mission.  The  maintenance  function  ensures  aircraft  and 
equipment  are  safe  serviceable,  and  properly  configured  to  meet  mission  needs  (AFI21-101, 
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2002).  To  prepare  the  aircraft  for  the  next  mission,  several  routine  maintenance  functions  are 
necessary.  Flight  line  maintenance  includes  processes  to  inspect,  service,  and  maintain  the 
aircraft.  During  the  aircrew  debriefing,  the  maintenance  personnel  are  looking  for  both  current 
faults  and  repeat  faults.  If  a  fault  is  confirmed  through  testing  or  inspection,  a  maintenance  crew 
heads  to  the  aircraft  to  conduct  a  repair  to  return  the  aircraft  to  mission-capable  status. 
Currently,  some  aircrews  report  the  aircraft  status  over  the  radio  or  “squawk”  discrepancies  back 
to  the  ground  crew  which  gives  them  a  head  start  on  the  repair  process.  The  current  Air  Force 
logistics  operations  system  is  reactive  in  nature,  meaning  that  after  the  aircraft  detects  a  part 
failure,  the  maintenance  person  must  perform  fault  isolation  procedures  and  then  steps  are  taken 
to  repair  or  replace  the  faulty  item.  This  process  is  usually  started  when  the  aircraft  detects  a 
fault,  which  sets  a  flag  and  sends  a  notice  to  the  pilot.  Depending  on  the  severity  of  the  fault  the 
mission  can  continue  and  the  information  is  saved  for  the  maintenance  debrief  (Faas,  2003).  The 
aircraft  are  prepared  for  flight  by  the  ground  crews,  the  pilots  load  the  assigned  mission,  take-off, 
perform  the  mission  and  land  to  complete  the  cycle. 

The  impetus  of  the  sortie  generation  process  is  the  sortie  schedule.  The  changing 
availability  of  the  aircraft  due  to  maintenance,  access  to  personnel  and  the  demands  of  operations 
cause  the  sortie  schedule  to  evolve  through  re-examination  and  alteration  at  regular  quarterly, 
monthly,  weekly  and  daily  intervals.  Meetings  are  held  between  operations  and  maintenance 
schedulers  to  negotiate  and  confirm  the  sortie  schedule  (Howard  and  Zaloom,  1 980).  The  sortie 
schedule  is  contrasted  against  the  yearly  plan  of  preventative  scheduled  maintenance,  periodic 
inspections,  Time  Compliance  Order  installations,  system  calibrations  or  Time  Change  Item 
replacements  (critical  parts  replaced  or  repaired  based  on  accumulated  flight  hours,  not  based  on 
part  condition).  The  quarterly  sortie  schedule  consists  of  a  forecast  of  mission  requirements  (i.e. 


16 


configuration,  needed  munitions  and  ordnance)  provided  by  operations  to  the  maintenance 
schedulers.  The  monthly  schedule  tentatively  assigns  specific  aircraft  to  specific  sorties. 
Weekly  planning  involves  assigning  takeoff  and  landing  times  for  a  given  sortie  and 
rescheduling  aircraft  due  to  constraints  that  arise  as  the  mission  requirements  solidify  (i.e.  sortie 
duration,  aircraft  configuration,  etc)  or  as  an  aircraft  becomes  non-mission  capable  (i.e. 
unscheduled  maintenance,  phase  inspection,  etc). 

One  of  the  key  maintenance  schedulers  is  the  Squadron  Production  Supervisor  (Pro 
Super).  The  weekly  matching  of  aircraft  to  sorties  is  a  key  responsibility  of  the  Pro  Super 
(AFI21-101,  2002).  The  weekly  meeting  of  maintenance  schedulers  is  a  time  used  to  review 
maintenance  progress  of  a  given  aircraft  through  the  flow  of  preventative  or  unscheduled 
maintenance.  The  Phase  Flow  is  a  term  used  to  describe  the  regular  maintenance  and  inspection 
on  aircraft  performed  in  accordance  with  their  engineered  lifecycle.  There  is  a  tradeoff  of 
accumulated  flight  hours  that  trigger  the  required  maintenance  (i.e.  Time  Change  Item 
replacements)  specified  in  the  Phase  Flow  and  the  hours  needed  to  complete  sorties.  A  Pro 
Super  wants  to  maximize  the  amount  of  sortie  hours  he  can  use  an  aircraft  before  he  loses  it  to 
the  phase  inspection  and/or  preventative  maintenance. 

4  User  Interaction 

The  goal  of  this  research  is  to  show  that  simulation  can  be  used  as  a  tool  to  assist  in  decision 
making  at  the  unit  level.  To  make  simulation  useful  at  this  level,  it  is  important  to  take  into 
consideration  the  users  and  their  interaction  with  the  simulation.  In  the  following  sections,  we 
review  the  literature  on  user  interaction,  examine  the  ways  in  which  users  interface  with 
simulation  models  and  describe  how  we  evaluated  the  user  interface  metaphor  or  prototype. 
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We  start  with  some  definitions.  The  Association  for  Computing  Machinery  Special  Interest 
Group  on  Computer-Human  Interaction  (ACM  SIGCHI)  in  1992  defined  human-computer 
interaction  in  this  way: 

“Human-computer  interaction  is  a  discipline  concerned  with  the  design,  evaluation  and 
implementation  of  interactive  computing  systems  for  human  use  and  with  the  study  of 
major  phenomena  surrounding  them.” 

In  the  mind  of  the  user,  the  interface  is  synonymous  with  the  system  (Hix  and  Hartson,  1993).  In 
fact,  Larson  (1992)  writes  the  interface  is  often  the  single  most  important  factor  in  determining 
the  success  of  a  system.  Yet  the  interface  is  just  one  part  of  the  whole  experience  a  user  has  with 
the  system.  The  user  experience  aims  to  maximize  ease  of  learning  and  memorability,  efficiency 
of  use,  and  subjective  satisfaction  while  minimizing  error  frequency  and  severity  in  the 
application  (Padilla,  2004).  Benyon  and  Murray  (1988)  differentiate  between  human-computer 
interaction  and  human-computer  interface.  Interaction  includes  all  aspects  of  the  environment 
such  as  the  working  practices,  office  layout,  provision  of  help  and  guidance  and  so  on.  The 
interface  is  the  part  of  a  system  with  which  the  user  comes  into  contact  physically,  perceptually, 
or  cognitively.  For  our  purposes,  the  user  experience  will  refer  to  way  in  which  the  decision¬ 
maker  interacts  with  the  user  interface  (UI)  and  the  actions  the  decision-maker  must  take  to  work 
through  the  problem  the  simulation  is  helping  to  solve  for  example  the  data  input  and  the  output 
analysis. 

4. 1  Decision  Support  Tools 

The  user  interface  metaphor  we  chose  to  purse  is  often  referred  to  as  a  decision  support  system. 
First  coined  by  Keen  and  Scott  Morton  (1978),  a  decision-support  system  (DSS)  is  any 
interactive  system  that  is  specifically  designed  to  improve  decision  making  of  its  users  by 
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extending  the  user’s  cognitive  decision-making  abilities  (Zachary  and  Ryder,  1997).  The  system 
can  take  many  forms  ranging  from  a  simple  table  to  elaborate  expert  systems.  For  military 
applications  of  DSS,  see  research  done  by  Buede  and  Bresnick  (1992),  Schraagen  (1997)  and 
Waag  and  Bell  (1997).  Traditionally,  decision  support  systems  fall  within  two  distinct  design 
viewpoints. 

4. 1. 1  Cognitive  Prosthesis 

A  cognitive  prosthesis  or  vehicle  decision  support,  tries  to  reduce  poor  decisions  by  eliminating 
the  defective  or  inconsistent  decision  making  of  a  person  by  replacing  the  decision-maker  with 
some  type  of  algorithm.  Some  examples  in  this  category  are  analytical  hierarchy  process 
(Tversky,  1972)  and  multi-attribute  utility  theory  (Keeney  and  Raiffa,  1976).  The  cognitive 
prosthesis  approach  is  often  used  for  routine  situations  where  decision  consistency  is  more 
important  than  the  selection  of  the  most  appropriate  response  under  unusual  circumstances.  This 
method  assumes  that  the  decision  follows  a  normative  model.  That  is  to  say  that  decision 
making  is  rational  and  when  provided  with  values  (costs  or  benefits)  associated  with  different 
exhaustive  choices,  mathematical  models  can  be  applied  to  those  values,  yielding  the  optimal 
choice  that  would  maximize  these  values  or  minimize  their  costs.  The  cognitive  prosthesis 
approach  proposes  and  imposes  specific  methodologies  to  the  decision  maker.  It  delivers  strong 
guidance  at  the  cost  of  flexibility. 

4.1.2  Cognitive  Toolbox 

The  other  traditional  perspective  referred  to  as  a  cognitive  toolbox  which  aims  to  support 
adaptive  human  decision  making  by  providing  supportive  instruments  rather  than  replace  the 
decision  maker.  As  the  name  implies,  this  could  be  a  toolbox  of  modeling  languages,  statistical 
functions,  graphing  packages  and  simulation  or  optimization  functions.  An  example  of  this 
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approach  would  be  a  spreadsheet  application  like  MS  Excel.  This  technique  is  not  associated 
with  any  normative  model  of  decision  making.  The  primary  objective  of  this  technique  is  to 
provide  a  set  of  tools  that  decision  makers  can  employ  in  structuring,  representing  and  exploring 
their  problems.  This  approach  provides  a  general  flexibility  in  that  it  enables  users  to  employ  a 
variety  of  approaches  and  tools  for  their  problem,  but  they  provide  little  guidance  on  both 
problem  representation  and  exploration. 

4.1.3  Conversational  Framework 

Our  research  focuses  on  yet  another  alternative  philosophy  that  is  starting  to  take  shape  in  the 
literature.  This  philosophy  is  often  referred  to  as  the  conversational  framework  (Figure  5)  for 
decision  support.  In  this  metaphor  the  decision-maker  interacts  with  an  autonomous  system 
module  or  stimulus  agent  in  which  different  kinds  of  knowledge  sources  can  be  encapsulated 
(Angehm,  1993),  for  example  the  rules  and  methodologies  of  the  sortie  generation  process.  The 
stimulus  agent  can  then  be  invoked  to  advise,  inform,  and  criticize  the  user  along  the  whole 
decision-making  process.  This  approach  is  rooted  in  the  assumption  that  decision  support  can  be 
provided  by  facilitating  and  stimulating  reflective  learning.  Facilitating  reflective  learning 
consists  of  enabling  decision  makers  to  incrementally  give  expression  to  their  views  on  the 
decision  at  hand  and  to  experiment  with  possible  solutions.  This  experimentation  allows  the  user 
to  create  alternative  viewpoints  and  insight  into  the  decision.  The  approach  allows  a  more 
transparent  decision  process  than  the  cognitive  prosthesis  and  a  more  guided  process  than  the 
cognitive  tool. 
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Figure  5:  The  Conversational  Framework  (Adapted  from  Angehrn,  1993) 

4.1.4  Review  UI  Issues  in  Simulation 

A  simulation  model  can  help  people  evaluate  their  current  working  hypothesis,  goals,  and  plans 
(Roth,  1994).  It  can  also  show  information  related  to  alternative  actions  such  as  resource 
requirements,  assumptions  and  required  configurations  (Rouse  and  Valusek,  1993).  The  military 
is  a  big  user  of  discrete  event  simulation  models  and  its  analysts  rely  heavily  upon  models  to 
gain  insight  into  the  wide  range  of  issues  facing  the  military  (Hill  et  al,  2001).  Based  on  these 
facts,  we  were  quite  comfortable  that  our  users  would  themselves  be  comfortable  with  idea  of  a 
simulation  as  a  stimulus  agent. 
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Simulation  by  itself  however  presents  some  UI  pitfalls.  In  terms  of  human  computer  interaction, 
many  studies  focus  in  on  animation  as  the  primary  tool  for  relating  information  from  the 
simulation  back  to  the  user  (Aiken  et  al,  1990).  These  so-called  visual  interactive  simulation 
(VIS)  systems  are  becoming  more  popular  due  mainly  to  fact  animation  software  is  more 
affordable  and  the  quality  of  the  output  has  increased  dramatically  (Carpenter  el  al,  1993). 
Kuljis  (1996)  argues  that  the  VIS  includes  better  validation,  increased  credibility  (and  hence 
model  acceptance),  better  communication  between  modeler  and  client,  incorporation  of  the 
decision  maker  into  the  model  via  interaction,  and  learning  via  playing  with  the  VIS.  The 
author  admits  there  is  little  empirical  evidence  to  support  these  claims.  Law  and  Kelton  (2000) 
note  that  animation  can  aid  in  enhancing  a  model’s  credibility  and  this  fact  alone  accounts  for  its 
expanding  use;  a  credible  model,  writes  Sargent  (1999),  is  a  model  that  decision  makers  are 
willing  to  use  because  they  have  confidence  in  the  model’s  results.  Supported  by  base  visits  and 
user  questionnaires,  we  knew  a  key  requirement  of  the  application  would  be  time.  Meaning,  this 
decision  was  made  quickly  and  the  DSS  could  not  hinder  that  process.  Sitting  through  an 
animation  of  the  simulation  would  prove  to  be  too  costly. 

Simulation  projects  tend  to  be  data-driven  and  not  necessarily  user-driven  Ironically,  most 
of  the  papers  on  simulation  systems  only  briefly  mention  the  data  input  capabilities  of  the 
systems,  if  at  all  (Kuljis,  1996).  According  to  user  questionnaires,  most  of  the  work  that  is 
devoted  to  creating  the  sortie  schedule  was  being  done  in  a  spreadsheet  application.  In  order  for 
the  simulation  model  to  work  effectively,  it  needs  real  time  data  from  the  user,  for  example,  the 
sortie  schedule  and  aircraft  availability.  Yet  we  did  not  want  to  add  more  time  to  the  decision 
process  by  forcing  the  user  to  manually  input  several  values  before  running  the  stimulus  agent. 
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4.1.5  Peer  Review  of  DSS 

To  get  a  better  picture  of  how  other  applications  deal  with  the  issues,  we  took  some  time  to 
review  some  peer  applications.  A  literature  reveals  there  are  several  domain  specific  tools  that 
have  been  developed  and  deployed.  One  such  example  is  Eagle  View  (Zahn  and  Renken,  1997) 
which  is  a  prototype  decision  analysis  tool  that  aids  wing  commanders  make  deployment 
decisions  based  on  current  state  of  the  base  based  on  status  of  planes,  supplies  and  manpower. 
The  Sortie  Generation  Rate  Model  (Harris,  2002)  aids  WG  Commanders  in  estimating  number  of 
sorties  that  can  be  flown  daily  given  fixed  input  parameters  to  aid  in  maximizing  utilization  rates. 
VMetric  is  a  commercial  application  that  incorporates  VARI-METRIC  mathematical  model  to 
aid  commercial  airlines  in  logistics  decisions  (Sherbrooke,  1986).  The  Scalable  Integration 
Model  for  Objective  Resource  Capability  Evaluations  (SIMFORCE)  was  built  by  the  Kelley 
Logistics  Support  Services  to  provide  wing  level  maintenance  managers  a  decision  support  tool 
for  conducting  what-if  problems.  McGee,  et  al  (2004)  developed  a  simulation-based  model  that 
was  constructed  for  the  (USAF)  MIME  repairable  parts  system.  The  authors  suggest  that 
through  the  development  of  simulation-based  models,  these  limiting  assumptions  can  be  relaxed, 
providing  a  model  that  captures  more  of  the  subtlety  and  variation  of  the  system.  This  model 
explores  the  privatization  of  the  transportation  practices  and  their  effects  the  USAF’s  supply 
chain  management.  Since  this  model  was  developed  just  to  experiment  with  transportation 
practices,  many  aspects  of  the  Sortie  Generation  Process  were  ignored  or  simplified  in  the  model 
design. 

4.2  Development  of  User  Interface 

This  section  of  the  report  presents  the  step-by-step  procedure  used  to  create  the  user  interface. 
We  knew  that  we  would  have  limited  access  to  active  production  supervisors.  There  are  a  small 
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number  of  production  supervisors  in  the  country  which  at  the  time  of  the  research  had  been 
dwindled  slightly  due  to  deployment  in  combat  missions.  To  help  develop  a  user-centered 
product  we  decided  to  adopt  a  scenario-based  design  strategy.  Introduced  by  Carroll  (2000),  this 
methodology  focuses  on  the  creation  of  day-in-the-life  scenarios  which  characterize  what 
happens  when  users  perform  typical  tasks.  The  idea  of  scenario-based  design  has  recently 
gained  in  popularity  (Rosson  and  Carroll  2002;  Nardi  1992;  Sonderegger,  Manning,  Charron  & 
Roshon,  2000).  The  approach  encourages  user  involvement  in  system  design,  provides  a  shared 
vocabulary  among  the  people  participating  in  the  system  development  project,  envisions  the 
uncertain  future  tasks  of  the  system  users,  and  enhances  ease  of  developing  instructional 
materials.  Furthermore  it  provides  a  good  brainstorming  tool  for  planning  and  allows  the 
stakeholders  to  consider  alternative  choices  in  decision-making. 

4.2.1  Scenario-based  Design  (Figure  6) 

Scenarios  illustrate  how  a  person’s  experiences  and  actions  unfold  in  specific  contexts  or 
situations.  They  also  can  highlight  interactions  (with  computer  systems,  people,  business  entities, 
etc.),  decision  processes,  activity  sequences,  influencing  factors  and  so  forth.  They  may  also 
illustrate  the  different  ways  in  which  varied  groups  or  types  of  people  experience  and  navigate 
through  similar  situations. 

Analysis  of  scenarios  can  foster  the  identification  of  areas  of  difficulty  (pain  points)  and 
experiential  gaps  (or  opportunities)  that  may  be  addressed  or  enhanced  through  technology 
solutions.  When  integrated  with  personas,  they  can  illustrate  how  different  target  audiences 
navigate  through  the  same  situation,  which  in  turn  can  suggest  ways  in  which  solutions  can  and 
should  be  adapted  for  varying  target  audiences.  Scenarios  can  serve  as  a  dynamic  reminder  of 
the  broad  range  of  contexts  and  situations  in  which  people  may  engage  an  interaction  with 
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technology  solutions,  fostering  thoughtful  discussions  about  how  to  design  solutions  that  fit  into 
and  complement  or  enhance  people’s  lives. 


USABILITY  SPECIFICATION 

Specific  posts  for  user 
performance,  satisfaction,  etc. 


DESIGN  RATIONALE 

Consequences  for 
stakeholders,  relevant 
issues  and  models 


SYSTEM  VISION 

Root  concepts  and 
motivation 


\ 


PROBLEM  SCENARIO 
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FUNCTIONAL  SPECIFICATION 

Scenario  actions  suggest 
required  system  functions 


OBJECT  MODEL 

Design  objects, 
responsibilities,  and 
collaborators 

Ul  METAPHOR 

Appearance  anti  behavior 
of  data  and  controls 


SUMMATIVE  EVALUATION 

Assess  with  respect  to  original 
goals  and  state  of  the  art 


DOCUMENTATION 
TRAINING  &  HELP 

Various  task-oriented 
support  artifacts 


FORMATIVE  EVALUATION 

Assess  project  progress  toward 
design  goals 


Figure  6:  Scenario-based  design  (Adapted  from  Go  and  Carroll,  2004) 

4.2.2  Design  Rationale 

The  purpose  of  decision  analysis  is  to  help  a  decision  maker  think  systematically  about  complex 
problems  and  to  improve  the  quality  of  the  resulting  decisions.  Typically  this  is  a  task  in  which 
1)  a  person  must  select  from  a  number  of  alternatives;  2)  there  is  some  amount  of  information 
available  with  respect  to  the  alternative;  3)  the  decision  timeframe  is  relatively  long  (i.e.  longer 
than  a  second);  4)  the  choice  of  alternatives  incorporates  some  uncertainty  which  means  the  best 
option  is  not  necessarily  apparent  (Wickens  et  al,  2004).  The  process  of  decision  making  can  be 
generally  represented  in  three  phases.  In  the  first  phase,  one  acquires  and  perceives  information 
or  cues  relevant  for  the  decision.  Secondly,  one  begins  to  generate  and  then  select  hypotheses  or 
situation  assessments  about  what  the  cues  mean,  regarding  the  current  and  future  state  relevant  to 
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the  decision.  Finally,  one  selects  the  alternative  to  take,  on  the  basis  of  the  inferred  state  and  the 
costs  or  values  of  different  outcomes.  By  definition,  decision  making  involves  risk  and  a  good 
decision  maker  effectively  assesses  risks  associated  with  each  alternative  (Medin  and  Ross, 
1992). 

4.2.3  Influence  Diagram 

To  help  us  get  a  better  picture  of  what  entities  are  involved  in  the  decision  to  change  the  sortie 
schedule  and  to  help  model  the  decision  process,  we  developed  an  influence  diagram  (Figure  7). 
An  influence  diagram  is  a  list  of  events  and  actions  with  arrows  showing  the  flow  of  information 
and  how  the  factors  influence  one  another.  An  arrow  leading  from  an  event  A  to  an  event  B 
implies  that  the  probability  of  obtaining  event  B  depends  on  whether  event  A  has  occurred.  An 
arrow  leading  from  a  decision  to  an  event  implies  that  the  probability  of  the  event  depends  on  the 
the  decision  stage.  An  arrow  leading  from  an  event  to  a  decision  implies  that  the 
knows  the  outcome  of  the  event  at  the  time  the  decision  is  made. 


choice  made  at 
decision  maker 
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Influence  Diagram 

Tail  Swap 


Resource 

Capacity 


Process 

Times 


Figure  7:  Influence  Diagram  for  the  Sortie  Generation  Process 
To  read  the  influence  diagram,  take  for  example  how  the  simulation  model  uses  the  input  for 
resource  capacity  to  describe  the  number  of  personnel  available,  AGE  available  and  parts 
available.  The  time  it  takes  to  replace  the  parts  by  the  personnel  and  using  required  AGE  is 
represented  by  the  Mean  Installation  Time.  Changes  to  this  event  could  dramatically  affect  the 
number  of  aircraft  available  to  the  flight  schedule,  thus  slight  variations  in  this  event  could 
dramatically  effect  the  decision  to  swap  aircraft  which  in  turn  would  affect  the  calculation  of  the 
risk  assessment  value. 

4.2.4  Key  Actors/Stakeholders 

A  key  step  in  the  decision  analysis  is  the  identification  of  the  key  stakeholders  that  influence  this 
decision.  According  to  AFI21-101  (2002)  and  interviews  held  with  an  AMU  at  Eglin  AFB,  the 
key  stakeholders  are: 
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Operations 

Operations  personnel  set  the  flight  schedule  for  training  and  combat  missions.  Operations 
personnel  also  assign  pilots  to  planes.  The  contact  meets  with  the  Pro-Super  weekly  to  help  set 
the  sortie  schedule. 

Maintenance  (AMU) 

•  Commander 

Ultimately  responsible  for  maintenance,  the  commander  approves  the  maintenance  plan 
at  all  levels  and  ensures  its  execution. 

•  Production  Supervisor  (Pro-Super) 

The  production  supervisor  is  responsible  for  the  maintenance  execution  down  to  the  job 
level.  The  production  supervisor  in  conjunction  with  the  Commander  and  Operations  is 
ultimately  responsible  for  the  weekly  flight  schedule.  The  Pro-Super’s  job  is  to  prioritize 
the  AMU’s  resources  in  order  to  execute  the  daily  maintenance  plan  and  flying  schedule. 

•  Maintenance  Crew  Chief 

The  maintenance  crew  chief  oversees  the  physical  hands  that  execute  repairs  on  the 
aircraft.  The  crew  chief  is  assigned  to  a  specific  plane  and  is  extremely  familiar  with  the 
nuances  of  its  maintenance. 

Though  there  are  several  stakeholders  in  the  decision,  this  research  and  therefore  the  DSS 
requirements  will  only  focus  on  the  production  supervisor. 

4.2.5  Baseline  Scenario 

The  focus  of  this  research  is  the  development  of  the  weekly  schedule  as  drafted  by  the 
production  supervisor.  The  baseline  scenario  was  developed  using  background  information  and 
process  descriptions  gathered  during  interviews  and  questionnaires.  MSgt.  MacNeece  is  not  a 
real  individual,  but  rather  he  is  aggregated  from  the  data  of  several  real  production  supervisors. 

To  prepare  for  the  morning  meeting,  MSgt.  MacNeece  likes  to  take  the  monthly 
plan  and  review  what  needs  to  happen.  He  transposes  this  information  into  MS 
Excel.  Then  he  logs  into  TASAMS  to  check  the  aircraft  availability  and  weekly 
schedule  of  maintenance.  To  get  a  better  picture  of  the  bottlenecks  that  might 
affect  the  maintenance  schedule,  he  checks  the  availability  of  resources  (i.e.  AGE, 
wash  house,  paint  bam)  and  calls  a  few  maintenance  chiefs  to  gauge  the 
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availability  of  personnel.  He  does  a  quick  sweep  of  the  aircraft  available  for  load 
training  to  make  sure  the  information  is  fresh  in  his  mind.  He  goes  back  into 
Excel  and  selects  the  cells  that  contain  the  tail  number  of  the  planes  he  has 
available;  he  bolds  the  numbers  to  make  the  numbers  stand  out.  He  does  a  quick 
calculation  of  the  current  wing  phase  flow  and  checks  TASAMS  again  for  the 
current  cannibalization  rate. 

The  scenario  was  then  used  to  identify  key  attributes  of  the  DSS  requirements  and  elements  of 
the  UI.  For  example,  the  use  of  MS  Excel  would  imply  a  comfort  in  the  Windows  graphical  user 
interface  therefore  UI  standards  developed  from  this  preexisting  familiarity  should  be  the 
baseline  for  our  interface  development.  Secondly,  the  production  supervisor  must  view 
information  from  several  different  sources  (i.e.  TASAMS,  monthly  plan)  therefore  a  core 
requirement  may  need  to  be  some  kind  of  data  import  to  help  combat  the  information  overload. 
4.2.6  Personas 

To  gain  further  insight  into  are  potential  users,  interviews  were  conducted  during  a  base  visit  to 
Eglin  AFB.  Questionnaires  were  also  sent  out  to  three  additional  production  supervisors.  Their 
backgrounds  and  means  of  drafting  the  weekly  schedule  were  surveyed.  They  were  aggregated 
together  to  form  an  archetypal  user  or  persona.  Personas  are  fictional  people  that  are  used  to  aid 
in  the  design  of  products  and  software  (Table  1).  The  term  “persona”  was  introduced  by  Alan 
Cooper  (1999).  Cooper  describes  a  scenario  as  “a  concise  description  of  a  persona  using  a 
software-based  product  to  achieve  a  goal.”  Prior  to  Cooper,  others  promoted  the  use  of  abstract 
representations  of  users  to  guide  design:  user  profiles  and  scenarios  derived  from  contextual 
inquiry  and  user  classes  fleshed  out  into  “user  archetypes”  (Beyer  and  Holtzblatt,  1998; 
Bloomberg,  Burrel,  and  Guest,  2003;  Hackos  and  Redish,  1998).  Personas  used  alone  can  aid 
design,  but  they  can  be  more  powerful  if  used  to  complement,  not  replace,  a  full  range  of 
quantitative  and  qualitative  methods  (Pruitt  and  Grudin,  2003). 
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Table  1:  Persona  Definition 


Background 

MSgt.  MacNeece  has  been  in  the  AF  for  2 1 
years.  In  that  time  he  has  worked  in  many 
positions  in  the  AMU.  He  knows  his  job 
well  and  the  jobs  of  the  men  who  serve 
under  him.  He  prides  himself  on  the  time 
he  spends  planning  maintenance  to  keep  the 
AMU  a  well  running  and  well  oil  machine. 

Quick  Stats 

Age:  39 

Education:  High  School  Diploma 
Skill  Level:  9 
Computer  Interest: 

Interested  in  computers  in  general  enjoys 
using  them 


Key  Insights 

Drafting  the  Weekly  Schedule 
MSgt.  MacNeece  likes  to  take  the 
monthly  plan  and  review  what  needs  to 
happen.  He  transposes  this  information 
into  MS  Excel.  Then  he  logs  into 
TASAMS  to  see  check  the  aircraft 
availability  and  weekly  schedule  of 
maintenance.  To  get  a  better  picture  of 
the  bottlenecks  that  might  affect  the 
maintenance  schedule  he  checks  the 
availability  of  resources  (i.e.  AGE,  wash 
rack,  paint  bam).  He  does  a  quick 
sweep  of  the  aircraft  available  for  load 
training  to  make  sure  it  is  fresh  in  his 
mind.  He  bolds  the  tail  numbers  of  the 
planes  he  has  available  and  does  a  quick 
calculation  of  the  current  wing  phase 
flow  and  checks  in  TASAMS  on  his 
current  cannibalization  rate. 

Accessing  Risk 

The  initial  calculation  of  the  phase  flow 
looks  to  be  about  5  points  higher  than 
MSgt.  MacNeece  is  used  to  seeing.  To 
help  gauge  why  the  phase  flow  is 
spiking  this  week  he  exports  wing  data 
from  TASAMS  and  imports  it  into  MS 
Excel.  He  walks  through  the  wizard  that 
pops  up  on  the  screen  and  adjusts  the 
installation  mean  time  up  slightly  to 
account  for  the  staff  shortage  in  the 
wash  rack.  The  output  report  shows  that 
he  should  expect  about  8  opportunities 
for  change,  which  is  about  4  more  than 
he  had  been  expecting. 


Computer  Experience: 

Modest,  not  an  expert  but  knows  how  to  use 
the  tools  he  needs 
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4.3  System  Vision 

After  having  identified  the  users  and  the  decisions  they  must  make,  the  next  step  was  to  lay  out 
our  vision  of  how  the  system  should  look  and  operate.  This  section  gives  a  review  of  the  system 
architecture,  human-computer  interaction,  and  basic  components  of  the  user  interface  metaphor. 

4. 3. 1  System  Architecture 

The  three  tier  architecture  (Figure  8)  is  a  popular  choice  for  web-based  applications  and  net- 
centric  information  systems  because  of  the  increased  performance,  flexibility,  maintainability, 
reusability,  and  scalability.  It  hides  the  complexity  of  distributed  processing  from  the  user.  The 
data  layer  provides  database  management  functionality  and  is  dedicated  to  data  and  file  services 
that  can  be  optimized  without  using  any  proprietary  database  management  system  languages. 
The  data  management  component  ensures  that  the  data  is  consistent  throughout  the  distributed 
environment  through  the  use  of  features  such  as  data  locking,  consistency,  and  replication.  It 
should  be  noted  that  connectivity  between  tiers  can  be  dynamically  changed  depending  upon  the 
user's  request  for  data  and  services.  The  business  tier  provides  process  management  services 
(such  as  process  development,  process  enactment  and  process  monitoring)  that  can  be  shared  by 
multiple  applications.  Our  DSS  has  three  basic  components,  the  input  analyzer,  the  simulation 
model  and  the  output  analyzer.  The  presentation  layer  provides  the  user  interface  to  these  two 
tiers  without  being  directly  tied  to  the  data  structure  or  the  process  management  services.  Thus 
the  interface  and  the  data  are  stored  separately  from  the  simulation  model  itself.  Therefore  the 
interface  can  be  altered  without  adverse  effects  to  the  model  or  the  underlying  data  structure. 
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Figure  8:  Sortie  Generation  Tool  Structure 

4. 3. 2  Agent  Interaction 

The  user  will  interact  with  the  system  in  two  cycles  (Figure  9).  The  first  cycle  will  capture  basic 
data  inputs  and  present  the  user  with  potential  change  opportunities  in  the  schedule.  These 
opportunities  are  times  within  the  simulation  when  a  plane  needs  to  be  swapped  out  of  the 
weekly  schedule  due  to  some  random  circumstance  such  as  unscheduled  maintenance.  The  user 
will  also  be  presented  with  a  risk  gauge,  which  is  the  calculation  of  variance  across  the 
simulation  run.  This  number  can  be  used  to  help  get  a  sense  of  how  metrics  such  as  utilization 
and  aircraft  availability  might  be  affected  by  the  swaps  suggested  in  the  simulation  model.  If 
risk  is  high,  the  decision  maker  can  go  through  a  second  cycle.  In  this  phase  of  the  decision 
process  the  user  can  tweak  the  values  for  installation  time  or  resource  capacity  to  see  if  variations 
in  those  numbers  might  affect  the  number  of  change  opportunities. 
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Figure  9:  Diagram  of  the  DSS  Cycle 

4. 3. 3  Functional  Requirements 

To  help  guide  the  interface  development  process,  we  turned  to  the  baseline  scenario  to  create 
four  core  functional  requirements  that  would  guide  the  user  interface  metaphor.  These 
requirements  represent  the  pain  points  or  those  areas  which  are  the  most  problematic  in  the 
decision  process.  Many  of  the  points  listed  below  piggy  back  off  of  the  design  heuristics  which 
will  be  describe  in  a  later  section. 

•  Reduce  Information  Overload 

The  weekly  schedule  is  generated  using  several  data  sources  (i.e.  TASAMS,  CAMS, 
monthly  plan,  crew  input)  yet  there  is  currently  no  means  to  view  all  this  information 
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in  an  aggregate  form.  For  a  weekly  schedule,  this  could  many  lines  of  data.  The 
system  should  aid  in  data  aggregation. 

•  Sensitivity  to  Resource  Limitations 

Due  to  budget  restrictions  and  aging  aircraft  parts  and  personnel  are  often  in  short 
supply.  As  resources  are  reduced  the  aircraft  availability  is  reduced.  The  system 
should  suggest  new  ways  to  work  within  these  constraints. 

•  Aid  the  Evaluation  of  Multiple  Schedules 

Due  to  the  overhead  needed  to  generate  a  schedule,  it  is  difficult  to  evaluate  multiple 
schedules  against  available  resources.  They  systems  should  allow  the  user  to  quickly 
manipulate  potential  schedule  and  resources. 

•  Sensitivity  to  Time  Constraints 

As  with  any  project,  timing  is  crucial.  The  DSS  should  not  dramatically  increase  the 
time  it  takes  to  generate  and  test  draft  sortie  schedules. 

5  Modeling  Approaches 

Once  the  data  handling  system  gathers  the  needed  inputs,  they  must  be  supplied  to  a  simulation 
model  which  will  provide  the  output  data.  This  section  of  the  report  will  describe  our  modeling 
methodology.  In  our  previous  research  we  stated  that  simulation  was  used  to  allow  the 
development  of  a  model  that  is  more  complete,  flexible,  and  expandable.  In  this  research  we 
take  advantage  of  the  expandability  of  a  simulation  model.  In  the  following  sections  we 
complete  a  brief  review  of  our  previous  model  and  continue  on  to  provide  a  detailed  description 
of  our  model  expansions. 

5.1  Simulation  Model 

A  baseline  model  was  developed  for  our  previous  research  in  Arena  ©  7.0  to  simulate  the  current 
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MIME  supply  chain  for  the  weapon  system  being  studied.  This  baseline  model  was  used  to 
compare  various  commercial  logistic  practices  that  potentially  could  be  adopted  by  the  Air  Force 
to  improve  supply  chain  efficiencies.  In  our  expanded  model  the  supply  chain  portion  of  the 
model  is  used  to  support  the  new  sortie  generation  logic. 

It  was  important  to  the  accuracy  of  our  results  that  our  model  be  a  close  representation  of 
the  current  repairable  parts  supply  chain  system.  Throughout  the  modeling  process,  both  during 
our  previous  research  and  in  the  development  of  model  expansions,  we  communicated  with  our 
contacts  at  the  Air  Force.  We  both  received  process  data  from  them  as  well  as  provided  them 
with  validation  statistics  and  model  data.  We  also  visited  Eglin  Air  Force  base  to  conduct 
interviews  with  maintenance  officers  and  get  a  firsthand  look  at  the  flight  line  in  operation.  This 
open  line  of  communication  allowed  us  to  gain  a  full  understanding  of  the  system  we  were 
modeling.  Appendix  1  contains  a  list  of  all  model  inputs  along  with  their  distributions  and 
parameters  which  were  developed  for  our  previous  research.  The  data  provided  supplied  in 
Appendix  1  was  developed  in  coordination  with  the  Air  Force  and  verified  through  our  contacts. 

5. 1. 1  Supply  Chain  Structure 

In  the  baseline  example  model,  there  are  six  independent  bases  supported  by  a  single  depot. 
There  are  twenty-four  aircraft  assigned  to  each  unit,  three  units  assigned  to  each  squadron,  and 
one  squadron  assigned  to  each  base.  In  this  structure,  there  are  a  total  of  72  aircraft  assigned  to 
each  base.  This  results  in  a  total  of  432  aircraft  within  the  system.  In  our  current  research  we  are 
only  interested  in  evaluating  the  performance  of  a  specific  number  of  aircraft  at  a  specific  base, 
but  the  other  bases  and  aircraft  are  important  to  simulate  competition  for  parts  throughout  the 
system.  The  six  bases  are  split  into  two  regions,  with  three  bases  in  each  region.  Figure  10 
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details  the  structure  for  the  baseline  model.  In  the  figure,  we  have  illustrated  the  squadron,  unit, 
and  aircraft  for  Base  1 .  The  other  bases  have  a  similar  structure. 


Figure  10:  Supply  Chain  Structure 

5.1.2  Weapon  Systems  and  Bases 

The  baseline  model  represents  weapon  systems  (for  the  purposes  of  this  research,  an  aircraft)  as 
objects  with  two  levels  of  indenture.  Initially,  each  aircraft  is  assigned  a  base  number,  index 
number,  and  tail  number.  The  index  number  is  a  model-wide  unique  number  assigned  to  each 
aircraft.  This  number  allows  the  user  to  compare  aircraft  individually  across  bases.  The  tail 
number  is  unique  to  each  aircraft  at  a  given  base.  The  base  number  indicates  the  base  at  which 
the  aircraft  is  stationed.  The  model  can  accommodate  a  variable  number  of  bases,  and  each  base 
can  have  a  variable  number  of  aircraft  (both  values  are  set  by  the  user 
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Each  weapon  system  has  two  levels  of  indentures.  The  first  level  of  indenture  entails 
aircraft  which  are  made  up  of  multiple  LRUs.  These  LRUs  are  in  turn  comprised  of  multiple 
SRUs  constituting  the  second  level  of  indenture.  The  number  of  SRUs  per  LRU  type  can  vary  as 
set  by  the  user;  however,  the  number  of  LRUs  per  aircraft  remains  constant  system  wide.  In  the 
baseline  model  there  are  six  individual  LRU  types.  Each  of  the  432  aircraft  in  the  system  is 
comprised  of  six  LRUs,  one  of  each  type.  LRUs  of  the  same  type  are  identical  and 
interchangeable. 

The  number  of  SRUs  per  LRU  can  vary  per  LRU  type  and  is  set  by  the  user.  .  The  SRU 
types  for  each  LRU  type  are  unique  and  cannot  be  shared  between  LRU  types,  but  within  the 
same  LRU  type,  the  component  SRUs  are  identical.  In  the  baseline  model  each  of  the  six  LRU 
types  are  comprised  of  four  SRUs  yielding  a  total  of  24  SRUs  per  aircraft. 

5.1.3  Weapon  Status 

In  our  model,  as  stated  earlier,  the  weapon  system  we  are  dealing  with  is  an  aircraft.  For  the 
purposes  of  this  model,  aircraft  are  always  categorized  as  being  in  one  of  three  states: 

(i)  Mission  Capable  (MC).  An  aircraft  is  designated  MC  when  it  is  capable  of  flying  a 
sortie.  This  status  can  correspond  to  an  aircraft  that  is  currently  flying  a  sortie  or  is 
waiting  to  be  assigned  to  a  sortie. 

(ii)  Non-Mission  Capable  (NMC).  An  aircraft  is  designated  NMC  when  one  or  more  of  its 
critical  SRUs  fails.  This  status  corresponds  to  an  aircraft  that  is  down  either  awaiting  a 
spare  part  or  currently  in  the  process  of  spare  part  installation.  NMC  aircraft  cannot  fly 
sorties. 

(iii)  Phase  Inspection  (PI).  An  aircraft  is  designated  PI  when  it  enters  the  phase  inspection 
module.  While  in  phase  inspection  the  aircraft  is  not  available  to  fly  sorties;  however,  the 
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aircraft  is  not  listed  as  NMC  because  phase  inspection  is  a  scheduled  maintenance 
operation. 

The  percentage  of  time  each  aircraft  is  in  each  state  is  tracked  and  reported  as  a  key  performance 
metric  of  the  simulation  model.  In  further  studies  the  number  of  weapon  system  states  could  be 
expanded  to  include  states  such  as  Partially  Mission  Capable,  Cannibalization,  etc. 

5.1.4  Failures 

The  failure  of  an  SRU  results  in  the  failure  of  an  LRU  and  therefore  the  weapon  system.  While 
on  the  aircraft,  SRUs  are  modeled  as  entries  in  a  2-dimensional  array.  Each  cell  of  the  TTF 
matrix  contains  the  time  to  failure  (TTF)  for  the  SRU  corresponding  to  that  cell.  This  value  is 
generated  by  a  distribution  held  in  the  expression  array  “mean-time-to-failure”  (MTTF).  Each 
cell  of  the  MTTF  expression  array  contains  the  distribution  used  to  generate  the  TTF  for  the  SRU 
corresponding  to  that  cell.  Currently,  the  baseline  model  contains  three  levels  of  MTTF  (in 
hours),  each  of  which  is  modeled  as  an  exponential  distribution  with  some  mean  value:  high- 
exponential  (500),  medium-exponential  (400),  and  low-exponential  (300).  There  are  eight  SRUs 
assigned  to  each  of  these  three  levels. 

While  an  aircraft  is  operational  it  accrues  operating  hours,  and  each  cell  corresponding  to 
that  aircraft  in  the  TTF  array  (i.e.  every  cell  representing  a  component  SRU  for  that  aircraft)  is 
decremented  equivalently.  Aircraft  failure  occurs  when  any  of  the  component  SRU  cells  equals 
or  drops  below  zero. 

5. 1. 5  Sortie  Assignments 

In  our  previous  research  sorties  were  simply  assigned  in  a  random  fashion  to  available  planes. 
This  research  concentrates  on  the  sortie  generation  process;  therefore,  a  significant  amount  of 
detail  was  added  to  the  sortie  assignment  process.  Sorties  for  a  specific  number  of  aircraft 
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determined  by  the  user  will  be  run  on  a  daily  basis  from  a  schedule  that  is  provided  by  the  user. 
The  schedule  includes  the  tail  number,  takeoff  time,  landing  time,  and  configuration  required  for 
each  sortie  as  described  in  the  section  describing  user  inputs.  At  the  beginning  of  each  day  a 
logic  entity  is  created  that  examines  the  schedule  and  creates  all  of  the  sortie  entities  for  the  day. 
The  sortie  entities  will  have  the  following  attributes:  tail  number,  time  until  release,  takeoff 
time,  sortie  length,  expected  landing  time,  and  configuration.  The  tail  number  specifies  which 
aircraft  is  supposed  to  execute  the  sortie.  Time  until  takeoff  (TUT)  is  a  new  term  defined  for  this 
simulation,  and  is  equal  to  the  Takeoff  time  minus  the  current  time,  yielding  the  delay  period 
before  the  sortie  is  scheduled  to  be  executed.  The  time  until  release  (TUR)  is  generated  from  the 
triangular  distribution  with  parameters  (TUT-2hr,  TUT-2.5hr,  TUT-3hr).  The  TUR  gives  us  the 
delay  between  the  current  simulation  time  and  the  time  at  which  the  aircraft  will  be  released  for 
pre-flight  operations.  Expected  sortie  length  (ESL)  is  defined  as  landing  time  minus  takeoff 
time.  The  sortie  length  is  generated  from  the  triangular  distribution  with  parameters  (ESL-0.5hr, 
ESL,  ESL+0.5hrs).  Once  the  sortie  entities  have  their  attributes  assigned,  they  proceed  to  a 
delay  block  where  they  are  delayed  the  specified  TUR.  When  the  sortie  is  released  it  sends  a 
signal  to  the  queue  holding  the  waiting  aircraft,  and  the  aircraft  proceeds  to  the  pre-flight 
operations. 

5.1.6  Pre-Flight  Operations 

The  aircraft  are  released  for  Pre-Flight  well  before  the  scheduled  sortie  time.  In  our  simulation 
we  generate  this  release  time  using  the  triangular  distribution  and  store  it  as  an  attribute  as 
described  in  the  previous  section.  The  first  operations  to  be  performed  once  the  aircraft  have 
been  released  for  pre-flight  are  refueling,  weapons  loading,  and  configuration.  The  time  required 
to  complete  the  refueling  and  weapons  loading  are  sampled  from  triangular  distributions  with 


39 


parameters  (8,  10,  12)  and  (25,  30,  35)  minutes  respectively.  If  the  aircraft  must  undergo  a 
configuration  change,  the  aircraft  is  delayed  by  a  processing  time  generated  from  the  triangular 
distribution  with  parameters  (30,  45,  60)  minutes.  Before  the  aircraft  proceeds  to  the  next  set  of 
preparatory  operations,  the  production  supervisor  must  do  an  exceptional  release  for  the  aircraft. 
This  exceptional  release  includes  a  general  walk  around  and  forms  check.  The  time  to  complete 
the  exceptional  release  is  generated  from  the  triangular  distribution  with  parameters  (8,  10,  15). 
At  this  point  in  our  model,  we  simulate  the  probability  that  an  error  with  the  aircraft  was  found 
during  the  preceding  operations  using  a  Bernoulli  distribution.  The  probability  that  an  error  was 
found  is  set  to  0.005,  while  the  probability  that  there  were  no  errors  is  set  to  0.995.  If  an  error 
was  found,  we  again  use  the  Bernoulli  distribution  to  model  the  probability  that  the  error  can  be 
fixed  before  the  aircraft  is  scheduled  to  fly  a  sortie.  The  probability  that  the  error  can  be  fixed  is 
set  to  0.50,  and  the  probability  it  cannot  be  fixed  is  set  to  0.50  as  well.  If  the  error  cannot  be 
fixed  in  the  remaining  time  there  are  two  alternatives:  if  there  is  a  spare  plane  available  it  is 
inserted  into  the  schedule,  if  there  is  not  a  spare  plane  available  the  sortie  is  cancelled.  If  the 
aircraft  is  error  free  or  has  been  fixed  in  the  allotted  time  it  is  deemed  crew  ready  and  moves  to 
final  preparations  that  include  engine  start,  final  systems  check,  and  taxiing.  The  final 
preparation  time  is  generated  from  a  triangular  distribution  (15,  20,  25)  minutes.  Final 
preparations  include  crew  show,  dash  one  check,  and  engine  start.  Since  the  aircraft’s  engines 
are  started  during  final  preparation  time,  the  aircraft’s  operating  hours  continue  to  accrue; 
therefore,  after  final  preparation  is  completed,  the  total  elapsed  time  since  engine  start  is 
decremented  from  the  TTF  values  associated  with  that  aircraft. 

A  pre-flight  check  of  all  aircraft  component  SRUs  is  then  performed.  If  any  of  the 
aircraft’s  component  SRUs  have  failed,  the  failed  part(s)  is/are  removed  from  the  aircraft  and 
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then  sent  to  the  repair  process.  The  aircraft  is  then  forced  to  wait  for  spare  parts.  This  is  called  a 
ground  abort.  If  a  ground  abort  occurs,  a  spare  is  substituted,  but  if  there  is  not  a  spare  available 
the  sortie  is  cancelled.  If  no  failures  are  found,  the  aircraft  flies  its  assigned  sortie. 

5. 1. 7  Sortie  Flights 

Once  the  aircraft  has  passed  the  pre-flight  inspection,  it  is  ready  for  takeoff.  Before  takeoff  the 
aircraft  must  first  taxi  to  the  runway.  The  time  it  takes  an  aircraft  to  taxi  to  the  runway  is 
generated  from  the  triangular  distribution  with  parameters  (5,  7,  10)  minutes.  At  this  point  the 
current  accrued  operating  hours  are  again  decremented  form  the  TTF  matrix  and  each  of  the 
aircraft’s  parts  are  checked  for  failure.  If  a  failure  is  found,  the  aircraft  proceeds  to  the  repair 
process.  If  there  is  a  spare  available  it  is  substituted,  and  the  sortie  is  delayed  by  the  taxi  time 
distribution.  If  there  is  not  a  spare  available  the  sortie  is  cancelled.  After  the  end  of  runway 
check,  the  aircraft  is  ready  for  takeoff.  The  time  it  takes  for  each  aircraft  to  takeoff  is  generated 
from  the  triangular  distribution  with  parameters  (2,  3,  4)  minutes;  however,  before  the  aircraft 
can  takeoff  it  must  first  wait  until  a  runway  is  available.  After  seizing  a  runway  and  taking  off, 
the  aircraft  undergoes  an  in-flight  check.  If  any  SRUs  are  found  to  be  failed,  the  sortie  is 
aborted,  a  runway  is  seized,  the  aircraft  lands,  and  then  moves  to  the  repair  process  as  previously 
described.  This  is  termed  an  air  abort.  If  the  aircraft  passes  the  in-flight  check,  it  continues  to 
fly  the  sortie. 

The  sortie  duration  is  part  of  the  user  input  captured  on  the  schedule,  and  is  captured  as 
an  attribute  as  described  in  section  6.1.5.  The  aircraft  is  delayed  by  the  previously  calculated 
sortie  duration.  After  completing  the  sortie,  the  aircraft  seizes  a  runway  and  lands.  Landing  time 
in  minutes  is  generated  from  a  triangular  distribution  with  parameters  (14,  15,  16).  Once  the 
aircraft  has  landed,  it  undergoes  its  post  flight  check.  The  duration  of  the  sortie  is  decremented 


41 


from  all  of  the  corresponding  cells  in  the  aircraft’s  TTF  matrix.  If  any  of  the  aircraft’s  SRUs  are 
found  to  be  failed,  the  failed  parts  are  removed  from  the  aircraft  and  proceed  to  the  repair 
process,  and  then  the  aircraft  moves  to  wait  for  spare  parts.  If  the  aircraft  passes  the  post-flight 
check,  it  will  continue  on  to  wait  for  its  next  scheduled  sortie.  Aircraft  will  continue  to  fly 
sorties  until  one  of  the  flight  checks  indicates  that  one  or  more  of  its  aircrafts  SRUs  has  failed. 

5.1.8  Phase  Inspections 

The  total  operating  hours  for  each  aircraft  is  tracked.  Once  an  aircraft  accrues  280-320  operating 
hours,  it  must  under  go  a  phase  inspection.  While  an  aircraft  is  in  the  phase  inspection  process 
its  weapon  system  status  is  set  to  PI.  A  phase  inspection  is  a  complete  check  of  the  aircraft  from 
top  to  bottom.  For  a  more  complete  description  of  our  modeling  of  the  phase  inspection  process 
please  reference  our  previous  report. 

5. 1. 9  The  Replacement  Process 

The  model  checks  each  SRU  associated  with  the  aircraft  sequentially  in  each  of  the  flight  checks. 
The  first  time  a  cell  in  an  aircraft’s  TTF  matrix  is  less  than  or  equal  to  zero,  the  aircraft  is 
considered  in  failure.  When  a  failed  SRU  is  detected,  the  aircraft  is  marked  as  being  failed,  and 
the  model  removes  the  SRU  from  the  aircraft.  This  SRU  is  sent  to  the  repair  process,  which  is 
described  later.  The  model  then  continues  to  check  for  other  SRU  failures  on  that  aircraft.  Once 
a  failed  aircraft  completes  the  flight  check,  the  model  performs  an  inventory  check  for  the  failed 
parts  associated  with  that  aircraft.  Once  a  needed  spare  part  is  available  at  the  base,  the 
installation  process  begins.  After  all  failed  parts  have  been  replaced  on  the  aircraft,  the  aircraft  is 
once  again  mission  capable. 
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5.1.10  Repair  Process 

When  a  SRU  is  deemed  defective,  the  model  creates  an  entity  representing  the  defective  SRU. 
It  is  highly  unlikely  that  this  failed  SRU  can  be  repaired  at  the  base  (Miller  1992).  In  the 
majority  of  cases,  the  SRU  must  be  sent  to  the  depot  for  repair.  If  the  SRU  can  be  repaired  at  the 
base,  the  travel  time  to  the  repair  station  from  local  inventory  is  assumed  to  be  zero  and  the  SRU 
enters  the  queue  for  the  base  repair  resource.  If  the  SRU  must  be  repaired  at  the  depot,  the  SRU 
is  delayed  some  shipping  time,  and  then  enters  the  queue  for  the  depot  repair  resource 

The  repair  stations  at  all  bases  and  the  depot  give  priority  to  backorders  for  repair  jobs. 
Repair  times  at  each  base  and  the  depot  are  random  distributions  set  by  the  user.  In  the  baseline 
model,  the  repair  times  are  generated  from  an  exponential  distribution  with  a  mean  of  eight 
hours. 

If  the  part  must  be  sent  to  the  depot  for  repair,  an  order  for  the  part  is  generated  and  sent 
to  the  depot.  This  order  waits  in  the  order  queue  at  the  depot  as  mentioned  earlier.  This  practice 
holds  with  a  one  for  one  inventory  policy.  In  other  words,  for  every  part  that  is  sent  to  the  depot 
an  order  is  generated  for  a  part  to  be  sent  back  to  the  base,  a  one-for-one  replenishment  policy. 
Again  in  this  queue,  backorders  are  given  priority. 

Upon  completion  of  the  repair  process,  the  SRU  becomes  functional  and  the  part  is  sent 
to  inventory.  If  the  SRU  was  repaired  at  the  base,  the  base  inventory  is  incremented.  If  the  part 
was  repaired  at  the  depot,  the  depot  inventory  is  incremented. 

5.2  Initializing  the  Model 

The  focus  of  the  sortie  generation  model  is  the  execution  of  a  weekly  schedule  at  the  squadron 
level.  In  our  previous  model,  we  focused  on  modeling  the  supply  chain.  In  our  sortie  generation 
model  we  expand  our  previous  model  to  include  a  detailed  modeling  of  the  sortie  generation 
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process.  The  supply  chain  and  its  multiple  demand  points  model  competition  for  parts 
exogenous  to  the  sortie  generation  process.  This  competition  for  parts  will  have  a  direct  effect 
on  the  ability  of  the  squadron  to  meet  a  specific  schedule. 

In  order  to  correctly  evaluate  a  schedule  provided  by  the  user  the  model  must  be 
initialized  to  a  current  state  before  schedule  execution.  To  capture  a  beginning  state  for  the 
supply  chain  the  model  is  warmed  up  for  a  6  month  period  of  time.  This  effectively  spreads 
parts  and  resources  throughout  the  supply  chain  giving  a  starting  point  for  executing  the  given 
schedule. 

In  many  cases  the  Pro.  Super,  has  some  knowledge  of  the  state  of  the  supply  chain.  That 
is  to  say  they  can  estimate  if  parts  are  expected  to  move  a  little  faster  or  slower  during  the 
scheduled  week.  This  is  effectively  modeled  by  allowing  the  user  to  adjust  the  number  of  parts 
within  the  system  by  plus  or  minus  10-20%  from  the  input  screen.  This  also  allows  the  user  to 
do  some  what  if  analysis. 

5.3  Initial  Aircraft  State 

After  the  warm-up  period  has  been  completed,  the  warm-up  aircraft  at  the  experimental  base  are 
deleted  from  the  model.  At  this  point  new  aircraft  are  created  with  initial  settings  as  defined  by 
the  user.  This  is  one  of  more  challenging  modeling  issues  related  to  utilizing  simulation 
technology  at  this  level.  Essentially,  we  must  determine  how  to  capture  the  beginning  state  of 
each  aircraft  within  the  squadron.  From  looking  at  Air  Force  documentation  and  observing  the 
system  we  have  developed  the  following  plan  for  capturing  the  initial  aircraft  state. 

5.3.1  Status 

In  this  case  we  are  using  status  to  refer  to  the  current  condition  of  the  aircraft,  much  like  the 
reports  handed  out  at  the  morning  meetings  between  the  Pro  Super  and  expediters.  To  capture 
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aircraft  status  a  dropdown  is  provided  which  includes  different  states.  This  dropdown  is 
accompanied  by  a  series  of  three  input  boxes  that  will  be  used  to  capture  the  expected  delay  time 
for  those  states  which  require  the  aircraft  to  be  incapable  of  flying  sorties.  These  three  boxes  are 
used  as  input  to  the  triangular  distribution,  high  value,  low  value,  and  mode.  The  states  that  are 
available  in  the  dropdown  are  as  follows: 

•  Mission  Capable  -  available  to  fly  sorties 

•  Phase  Inspection 

•  Unscheduled  Maintenance 

•  Scheduled  Maintenance 

•  Depot  Maintenance 

•  Cross  Country 

•  Cannibalization 

•  Other 

In  many  situations  involving  maintenance  there  will  be  multiple  items  that  must  be  repaired.  To 
capture  this  scenario,  for  example,  there  could  be  5  sets  of  input  boxes  for  the  expected  delay 
time.  These  boxes  will  be  accompanied  by  an  input  box  for  the  cause  and  a  dropdown  box  to 
specify  the  processing  order.  The  cause  input  will  have  no  effect  on  the  model,  but  will  help  the 
users  track  what  they  have  entered.  The  processing  order  dropdown  box  will  include  the  number 
1-5,  and  the  work  delay  times  that  are  entered  will  be  experienced  according  the  processing  order 
number  selected  for  that  delay.  To  model  concurrent  operations  the  same  processing  order 
number  can  be  selected  for  multiple  delays.  In  other  words  when  the  model  is  run  it  will  process 
the  delays  for  each  aircraft  according  the  processing  order.  If  multiple  delays  bear  the  same 
processing  order  number,  the  longest  of  the  delays  will  be  experienced  by  the  aircraft. 
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5.3.2  Configuration 

Configuration  describes  the  physical  configuration  of  the  aircraft.  In  the  case,  we  observed  at 
Eglin  AFB  that  the  only  physical  configuration  concern  was  the  tank  configuration.  Thus,  in  this 
proof  of  concept  we  only  model  this  configuration  under  the  recognition  that  other  weapon 
systems  and  missions  may  require  additional  configuration  modeling.  The  tank  configuration  is 
captured  through  a  dropdown  box.  The  tank  configurations  we  observed  are  as  follows: 

•  One  Tank 

•  Two  Fuel  Tanks 

•  Three  Fuel  Tanks 

•  Clean 

Different  missions  require  different  tank  configurations  as  specified  by  the  schedule.  Therefore 
it  is  important  to  capture  the  initial  configuration  to  correctly  model  the  delay  in  changing 
configuration  when  needed. 

5.3.3  Phase  Hours 

The  phase  flow  is  a  major  consideration  taken  into  account  when  generating  a  schedule.  It  is 
also  important  for  the  Pro  Super  to  know  where  they  will  stand  after  executing  a  particular 
schedule.  Phase  hours  indicate  the  number  of  flying  hours  an  aircraft  must  accumulate  before  its 
next  phase  inspection.  In  this  regard,  it  is  an  indication  of  the  number  of  hours  accrued  by  the 
aircraft  since  its  last  full  inspection,  and  therefore,  it  is  an  indication  of  the  state  of  the  aircraft’s 
working  parts.  The  phase  hours  for  each  aircraft  can  be  captured  through  an  input  box.  The 
Mean  Time  to  Failure  for  all  of  the  aircraft’s  parts  can  then  be  derived  from  the  phase  hours.  To 
initialize  the  TTF  matrix,  the  TTF  for  each  SRU  is  generated  as  previously  described,  and  then 
the  accrued  phase  hours  are  subtracted  from  the  generated  TTF  values. 
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5.4  Model  Output  Data 

For  this  model  to  be  useful,  the  data  provided  to  the  user  in  the  end  must  be  both  easy  to 
understand  and  interpret.  For  our  model  we  have  developed  two  metrics:  deviation  from  45° 
phase  flow  and  number  of  change  opportunities. 

In  the  USAF  maintenance  officers  are  called  to  keep  their  aircraft  in  a  continual  state  of 
readiness.  One  of  the  main  measures  of  readiness  is  adherence  to  a  45°  phase  flow  line.  Phase 
flow  refers  a  graph,  across  a  squadron,  of  the  number  of  operational  hours  remaining  before  each 
aircraft  must  undergo  a  phase  inspection,  and  therefore,  be  out  of  operation  for  a  number  of  days. 
On  this  graph  it  is  important  that  there  are  not  a  large  number  of  aircraft  that  are  very  close  to 
phase  inspection,  and  conversely  it  is  important  that  there  are  not  a  large  number  of  aircraft  with 
just  a  few  accumulated  hours.  If  there  are  a  large  number  of  aircraft  at  either  end  of  the  spectrum 
it  would  mean  that  a  large  number  of  aircraft  would  come  due  for  phase  inspection  at  basically 
the  same  time.  In  essence  an  unbalanced  phase  flow  could  render  a  large  portion  of  the  squadron 
useless  at  a  critical  time.  Maintenance  managers  attempt  to  keep  an  even  dispersion  of  phase 
hours  across  the  squadron.  The  perfect  dispersion  would  result  in  a  45°  line.  After  our  model 
has  run  the  user  provided  schedule  it  will  output  the  number  of  phase  hours  for  each  aircraft  and 
graph  them.  The  percent  deviation  from  a  45°  line  is  then  calculated  and  displayed  as  a  measure 
of  schedule  performance. 

There  are  many  instances  when  executing  a  schedule  in  which  problems  will  arise  that 
can  be  handled  by  the  maintenance  officers  in  a  manner  which  does  not  result  in  a  schedule 
deviation.  In  our  model,  instead  of  trying  to  mimic  the  behavior  of  the  maintenance  staff 
perfectly,  we  keep  tack  of  instances  in  which  the  maintenance  officer  would  have  had  to  interact 
with  the  schedule.  We  call  these  instances  change  opportunities.  For  example,  if  an  error  occurs 
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with  the  aircraft  before  exceptional  release  the  Pro  Super  might  tell  his  maintenance  crew  to 
concentrate  their  efforts  on  fixing  that  aircraft  before  it  is  scheduled  to  fly,  or  the  error  might  be 
serious  enough  that  a  spare  aircraft  must  take  its  place.  In  this  situation,  there  are  a  number  of 
things  that  might  occur  so  for  the  purposes  of  our  model  any  aircraft  error  results  in  a  change 
opportunity  or  an  opportunity  for  the  maintenance  officers  to  interact  with  the  schedule. 

5.5  Data  Handling  Approaches 

In  our  previous  model  data  was  handled  strictly  through  Excel.  The  user  could  change  values  in 
Excel  which  were  then  imported  into  the  model  as  simulation  parameters.  For  our  current 
research  Excel  is  still  used  as  the  data  capturing  mechanism.  We  felt  that  the  end  user  would  be 
most  comfortable  using  the  spreadsheet  format.  In  our  new  model  once  the  data  is  captured  in 
Excel  it  is  imported  into  Access  and  from  there  we  bring  it  in  to  the  simulation.  We  chose  to  use 
Access  as  an  intermediate  step  because  we  want  our  model  to  be  scalable  to  handle  large 
amounts  of  data.  For  example  in  the  future  it  may  be  useful  to  model  a  larger  number  and  more 
complicated  of  LRUs.  This  would  require  a  large  amount  of  data  for  failure  rates,  repair  times, 
etc.  There  are  many  instances  where  large  amounts  of  data  might  be  useful  in  the  future  and 
Access  allows  us  to  have  the  capability  to  handle  the  data  efficiently. 

6  Test  Scenario  Description 

In  this  section  we  walk  through  the  user  interface  developed  for  this  tool,  and  then  develop  a 
schedule  to  test  our  tool.  We  demonstrate  the  usability  of  our  tool  as  well  as  provide  real  results. 
The  schedule  presented  below  was  developed  from  actual  USAF  schedules  and  has  the 
complexity  to  test  every  piece  of  our  tool. 


48 


6.1  User  Interface  Metaphor/Prototype 

This  section  will  describe  the  user  interface  in  detail  including  screen  shots  and  detailed 
descriptions  of  the  inputs  (Figures  11-19).  Appendix  1  contains  all  of  the  inputs  which  were 
used  in  our  previous  research  and  are  still  available  in  this  expanded  model.  To  facilitate  the 
core  requirements,  we  decided  the  best  interface  metaphor  is  a  wizard.  The  user  is  walked 
through  the  cycles  of  the  DSS. 

6.2  First  Cycle 

The  following  sections  will  describe  the  first  cycle  in  the  DSS  cycle.  Each  of  the  steps  is 
illustrated  with  a  screenshot  followed  by  a  description  of  the  screenshot. 

6.2.1  Step  One 


Figure  11:  Welcome  Screen 

This  screen  is  the  wizard  welcome  screen  and  describes  what  the  user  can  expect  by  completing 
the  steps  within  the  wizard.  It  allows  the  user  to  denote  if  he/she  would  prefer  to  use  the 
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simulation  models  fallback  distributions  for  the  maintenance  time  and  installation  time.  From 
the  screen,  the  user  can  travel  in  several  directions.  Clicking  the  “SETTINGS”  button  will  take 
the  user  to  the  settings  window  defined  on  the  next  page.  Clicking  “NEXT”  will  take  the  user  to 
the  sortie  schedule  window.  Clicking  “CANCEL”  will  close  the  application.  “BACK”  is 
disabled. 


Tail  Swap  Risk  Analysis  Wizard 


Wizard  Settings 

Change  the  default  settings  for  the  wizard. 


*  •  Simulation  Set-up 
Number  of  Echelons  F 

Number  of  Bases  pi 

Number  of  LRUs  pS 

Maintenance  Data  Defaults  — —  -  - 
Installation  Time  |TRiA(12, 15,13.6) 


Repair  Time  |TRIA(12,15,13.6)  | 

Recovery  Time  [  t RIA(1 2T1 5  1 3.6) 

Launch  Time  |tRIA(12,15,13  6) I 

Use  maintenance  data  defaults. 


Number  of  SRUs  ^3 


•  SRU  Failure  Rate . -  . - 

LRU  Rate  (lamda) 

1  j  12  per  hour 

|  per  hour 
per  hour 


3  3 


Apply  to  All  LRUs 


Save  |  Cancel 


Figure  12:  Wizard  Settings 

The  “SETTINGS”  button  appears  at  the  bottom  left  of  all  the  wizard  screens.  From  this  menu 
the  user  can  change  the  simulation  models  default  settings.  Once  the  “SAVE”  button  or  the 
“CANCEL”  button  is  clicked  the  user  is  returned  to  the  welcome  screen.  Clicking  “?”  in  the  top 
right  comer  will  take  the  user  to  help  section. 
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6.2.2  Step  Two 


Input  Sortie  Schedule 

Import  the  data  or  enter  manually. 


—Import  Data — - 

Select  the  file  on  your  hard  drive. 


Browse 


Enter  the  sortie  schedule  for  the  week  to  be  analyzed. 


Day 

T  urns 

Sunday 

OFF 

Monday 

12/10 

T  uesday 

12/10 

Wednesday 

12/10 

Thursday 

12/10 

Friday 

1Q/B 

Saturday 

1Q/0 

. . . 

SM?  Sill"  :  - 


:'r;vb'4Hi 
:  *'■ 


Settings 


„'V.  1^^.'.: 


••  •••.<  \  "V- 

<Back  1  Next> 


r  >v>;;  w  .  af) 

Cancel 


Figure  13:  Input  Sortie  Schedule 

This  screen  allows  the  user  to  import  the  sortie  schedule  or  enter  the  schedule  manually.  If  the 
schedule  is  imported  the  user  will  have  the  option  to  tweak  the  data  once  it  is  loaded  into  the 
wizard  and  displayed  in  the  schedule  text  area.  The  DAY  column  refers  to  the  day  of  the  week 
on  which  this  event  will  occur.  The  TURNS  column  refers  to  the  number  of  aircraft  to  be 


launched  in  the  moming/evening.  Clicking  the  “SETTINGS”  button  will  take  the  user  to  the 
settings  window  defined  on  the  next  page.  Clicking  “NEXT”  will  take  the  user  to  the  aircraft 
status  window.  Clicking  “BACK”  will  take  the  user  back  to  the  welcome  screen.  Clicking 
“CANCEL”  will  close  the  application. 
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6.2.3  Step  Three 


Tail  Swap  Risk  Analysis  Wizard 


Input  Aircraft  Availability  Data 

Import  the  data  or  enter  manually. 


•—Import  Data- . *r~~; - - 

Select  the  file  on  your  hard  drive.  View  Example, 


Enter  the  aircraft  availability  for  the  week  to  be  analyzed. 


|  Tail  i  Configuration 

Status 

A 

s  125Q3B  Alpa  Available 

|  125037  Alpa  CC 

125Q3B  Beta 

Depot 

125039  Beta 

Available 

125040  Beta 

Available 

125041  Gamma 

Available 

125042  Beta 

Available 

▼I 

■  .  .. 

Settings  J 

<  SackT"1’] 

Next> 


Browse 


.tTiWju 


Cancel 


Figure  14:  Input  Aircraft  Availability 

This  screen  allows  the  user  to  import  the  aircraft  availability  or  enter  the  aircraft  status  manually. 
If  the  aircraft  status  is  imported  the  user  will  have  the  option  to  tweak  the  data  once  it  is  loaded 
into  the  wizard  and  displayed  in  the  aircraft  status  text  area.  The  TAIL  column  refers  to  the  tail 
number  used  to  identify  a  specific  weapons  system.  The  CONFIGURATION  column  refers  the 
fuel  tanks  and  ordnance  configuration  the  aircraft  currently  equipped  to  accept.  The  STATUS 
column  denotes  if  the  aircraft  is  available  to  the  schedule.  Clicking  the  “SETTINGS”  button  will 
take  the  user  to  the  settings  window  defined  on  the  next  page.  Clicking  “NEXT”  will  take  the 
user  to  the  maintenance  data  window.  Clicking  “BACK”  will  take  the  user  back  to  the  sortie 
schedule  screen.  Clicking  “CANCEL”  will  close  the  application. 
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6.2.4  Step  Four 


Input  Maintenance  Data 

Import  the  data  or  enter  manualy . 


. — Import  Data . . —  — 

Select  the  file  on  your  hard  drive. 


Enter  the  maintenance  times  In 


Average 


Installation 


Recovery 


Launch 


Figure  15:  Input  Maintenance  Data 

This  screen  allows  the  user  to  import  the  maintenance  data  to  be  used  by  the  simulation  model. 
The  system  uses  the  triangular  distribution  to  model  each  of  the  events.  Therefore  the  screen 
solicits  the  maximum,  minimum,  and  average  time  period  for  each  event.  This  information  can 


either  be  imported  or  manually  entered.  If  the  data  is  imported  the  user  will  have  the  option  to 
tweak  the  data  once  it  is  loaded  into  the  wizard  and  displayed  in  the  maintenance  text  area.  The 
TYPE  column  refers  to  the  event  in  the  sortie  generation  process.  The  HIGH  column  refers  to 
maximum  time  value  for  the  event.  The  LOW  column  refers  to  minimum  time  value  for  the 
event.  The  AVERAGE  column  refers  to  mean  time  value  for  the  event.  Clicking  the 
“SETTINGS”  button  will  take  the  user  to  the  settings  window  defined  on  the  next  page. 
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Clicking  “NEXT”  will  take  the  user  to  the  simulation  settings  window.  Clicking  “BACK”  will 
take  the  user  back  to  the  aircraft  availability  screen.  Clicking  “CANCEL”  will  close  the 
application. 

6.2.5  Step  Fixe 


Figure  16:  Simulation  Settings 

This  screen  allows  the  user  to  slightly  modify  the  distributions  from  the  maintenance  data  screen. 
During  the  first  cycle  of  the  application,  the  user  is  encouraged  not  to  change  the  settings  on  this 
screen.  Clicking  the  “SETTINGS”  button  will  take  the  user  to  the  settings  window  defined  on 
the  next  page.  Clicking  “NEXT”  will  take  the  user  to  the  simulation  output  window.  Clicking 
“BACK”  will  take  the  user  back  to  the  maintenance  data  screen.  Clicking  “CANCEL”  will  close 
the  application. 
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6.2.6  Step  Six 


Figure  17:  Output  Data 

This  screen  provides  the  user  with  the  simulation  output.  To  help  gauge  the  risk  to  the  aircraft 
availability  and  the  utilization  rate  the  risk  gauge  is  shown.  The  gauge  shows  the  average 
number  of  change  opportunities  the  simulation  model  has  found  along  with  the  goal  value.  The 
raw  simulation  data  is  also  displayed  in  the  textbox  below.  The  phase  flow  angle  is  shown 
across  the  simulation  runs,  based  on  the  variance  across  these  runs  the  risk  value  is  calculated. 
Clicking  the  “SETTINGS”  button  will  take  the  user  to  the  settings  window  defined  on  the  next 
page.  Clicking  “NEXT”  will  take  the  user  to  the  simulation  settings  window.  Clicking  “BACK” 
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will  return  the  user  to  the  simulation  settings  windows.  Clicking  “CANCEL”  will  close  the 
application. 

6.3  Second  Cycle 

The  following  sections  will  describe  the  second  cycle  in  the  DSS  cycle.  Each  of  the  steps  is 
illustrated  with  a  screenshot  followed  by  a  description  of  the  screenshot. 

6.3.1  Step  One 


Figure  18:  Modify  Simulation  Settings 

This  screen  allows  the  user  to  slightly  modify  the  distributions  from  the  maintenance  data  screen. 
During  the  second  cycle  of  the  application,  the  user  modifies  the  simulation  settings  to  see  if 
minor  variations  will  help  in  the  number  of  change  opportunities  or  improve  the  phase  flow 
angle.  Clicking  the  “SETTINGS”  button  will  take  the  user  to  the  settings  window  defined  on  the 
next  page.  Clicking  “NEXT”  will  take  the  user  to  the  simulation  output  window.  Clicking 
“BACK”  will  take  the  user  back  to  the  maintenance  data  screen.  Clicking  “CANCEL”  will  close 
the  application. 
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6.3.2  Step  Two 


Figure  19:  New  Simulation  Output 

This  screen  provides  the  user  with  the  simulation  output.  If  the  metrics  are  not  significantly 
improved  the  user  can  select  the  “BACK”  button  to  return  to  the  schedule  window  to  try  a 
different  schedule.  If  the  user  is  comfortable  with  the  results  the  schedule  can  be  noted  and  that 
draft  of  the  sortie  schedule  is  selected.  Clicking  the  “SETTINGS”  button  will  take  the  user  to  the 
settings  window  defined  on  the  next  page.  Clicking  “NEXT”  will  take  the  user  to  the  simulation 
settings  window.  Clicking  “CANCEL”  will  close  the  application. 
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6.4  Formative  Evaluation 
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Because  there  was  so  little  access  to  potential  users,  the  system  was  evaluated  for  usability  using 
an  inspection  method  known  as  heuristic  evaluation.  In  order  to  perform  the  evaluation,  a  set  of 
guidelines  or  heuristics  must  be  established. 

6. 4. 1  Design  Heuristics 

There  are  several  recognized  heuristics  in  the  literature  (Nielsen  and  Molich,  1990;  Shneiderman, 
1998).  The  most  common  are  listed  below: 

•  Visibility  of  system  status 

The  system  should  always  keep  users  informed  about  what  is  going  on,  through 
appropriate  feedback  within  reasonable  time. 

•  Match  between  system  and  the  real  world 

The  system  should  speak  the  users'  language,  with  words,  phrases  and  concepts  familiar 
to  the  user,  rather  than  system-oriented  terms.  Follow  real-world  conventions,  making 
information  appear  in  a  natural  and  logical  order. 

•  User  control  and  freedom  * 

Users  often  choose  system  functions  by  mistake  and  will  need  a  clearly  marked 
"emergency  exit"  to  leave  the  unwanted  state  without  having  to  go  through  an  extended 
dialogue.  An  “undo”  and  “redo”  functionality  will  be  necessary  for  this  reason. 

•  Consistency  and  standards  * 

Users  should  not  have  to  wonder  whether  different  words,  situations,  or  actions  mean  the 
same  thing.  Follow  platform  conventions. 
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•  Error  prevention  * 

Even  better  than  good  error  messages  is  a  careful  design  which  prevents  a  problem  from 
occurring  in  the  first  place. 

•  Recognition  rather  than  recall  * 

Make  objects,  actions,  and  options  visible.  The  user  should  not  have  to  remember 
information  from  one  part  of  the  dialogue  to  another.  Instructions  for  use  of  the  system 
should  be  visible  or  easily  retrievable  whenever  appropriate. 

•  Flexibility  and  efficiency  of  use 

Accelerators  --  unseen  by  the  novice  user  —  may  often  speed  up  the  interaction  for  the 
expert  user  such  that  the  system  can  cater  to  both  inexperienced  and  experienced  users. 
Allow  users  to  tailor  frequent  actions. 

•  Aesthetic  and  minimalist  design 

Dialogues  should  not  contain  information  which  is  irrelevant  or  rarely  needed.  Every 
extra  unit  of  information  in  a  dialogue  competes  with  the  relevant  units  of  information 
and  diminishes  their  relative  visibility. 

•  Help  users  recognize,  diagnose,  and  recover  from  errors  * 

Error  messages  should  be  expressed  in  plain  language  (no  codes),  precisely  indicate  the 
problem,  and  constructively  suggest  a  solution. 

•  Help  and  documentation 

Even  though  it  is  better  if  the  system  can  be  used  without  documentation,  it  may  be 
necessary  to  provide  help  and  documentation.  Any  such  information  should  be  easy  to 
search,  focused  on  the  user's  task,  list  concrete  steps  to  he  carried  out,  and  not  be  too 
large. 
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^Denotes  heuristics  that  appear  in  both  Shneiderman  and  Nielsen  and  Molich; 
otherwise  heuristics  are  those  suggested  by  Nielsen  and  Molich  alone. 

Due  to  the  nature  of  the  application,  we  felt  it  necessary  to  include  a  few  additional  heuristics  of 
our  own  devising.  They  are  as  follows: 

•  Transparency 

In  expert  level  systems  like  a  DSS,  users  need  to  know  how  figures  were  calculated.  If 
calculation  is  made  allow  the  user  to  drill  down  to  see  the  root  equation. 

•  Contextual  recognition  of  agent  output 

Simulation  output  numbers  can  overwhelm  the  user  if  they  are  not  put  into  context  of  the 
problem.  Present  figures  so  that  they  can  be  easily  mapped  back  into  the  decision 
context. 

•  Data  import  rather  than  data  input 

To  get  an  accurate  simulation  run,  real  world  data  is  needed  to  create  distributions  and 
calculate  means.  Whenever  possible,  allow  the  user  to  input  such  values  en  lieu  of 
making  them  hand  code  them  back  into  the  simulation  model. 

6.4.2  Heuristic  Evaluation 

Heuristic  evaluation  (Nielsen  and  Molich,  1990;  Nielsen  1994)  is  a  usability  engineering  method 
for  finding  the  usability  problems  in  a  user  interface  design  so  that  they  can  be  attended  to  as  part 
of  an  iterative  design  process.  Typically,  a  small  set  of  evaluators  examines  the  interface  and 
judges  its  compliance  with  recognized  usability  principles  (the  "heuristics").  The  inspection  of 
the  interface  is  performed  by  the  evaluator  alone.  During  the  evaluation  session,  the  evaluator 
goes  through  the  interface  several  times  and  inspects  the  various  dialogue  elements  and 
compares  them  with  a  list  of  recognized  usability  principles  (the  heuristics).  In  addition  to  the 
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checklist  of  general  heuristics  to  be  considered  for  all  dialogue  elements,  the  evaluator  obviously 
is  also  allowed  to  consider  any  additional  usability  principles  or  results  that  come  to  mind  that 
may  be  relevant  for  any  specific  dialogue  element.  Furthermore,  it  is  possible  to  develop 
category-specific  heuristics  that  apply  to  a  specific  class  of  products  as  a  supplement  to  the 
general  heuristics.  A  supplementary  list  of  category-specific  heuristics  was  also  provided  to  the 
evaluators.  After  all  evaluations  had  been  completed,  the  evaluators  allowed  to  communicate 
with  each  other  and  have  their  findings  were  aggregated.  This  procedure  is  important  in  order  to 
ensure  independent  and  unbiased  evaluations  ftom  each  evaluator. 

6.5  Test  Scenario  Inputs 

This  section  will  describe  the  data  which  was  used  in  our  test  scenario.  Table  2  describes  the 
availability  of  aircraft.  This  table  outlines  the  data  described  in  section  5.3  used  to  initialize  the 
state  of  the  aircraft  preceding  the  execution  of  the  schedule.  There  are  four  main  variables  that 
define  an  aircrafts  state  tail  number,  availability,  delay  until  ready,  phase  hours,  and 
configuration. 
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Table  2:  Initial  Aircraft  State 


1  Tail  Number 

Availability 

Dciay(hours) 

Hours  Until  Phase 

Config. 

010101 

Available 

0 

56 

Clean 

010102 

Available 

0 

57 

Clean 

010103 

Available 

0 

7 

Clean 

010104 

SM 

Triangular(l  2,1 5,20) 

17 

Clean 

010105 

Available 

0 

246 

Beta 

010106 

Available 

0 

191 

Clean 

010107 

xco 

T  riangular(24,25 ,27) 

94 

Gamma 

010108 

xco 

Triangular(24,25,27) 

95 

Gamma 

010109 

Available 

0 

96 

EXia 

010110 

DEMO 

Triangular(  1 69, 1 75, 1 80) 

102 

Alpha 

010111 

Available 

0 

51 

Beta 

010112 

CANN 

Triangular(200,224,280) 

30 

Alpha 

010113 

SM 

Triangular(80,84,9 1 ) 

134 

Clean 

010114 

DEMO 

Triangular(  1 2, 1 5,20) 

115 

Clean 

010115 

NMC 

Triangular(50,58,60) 

144 

ES391 

010116 

Available 

0 

167 

EB3IE1H 

010117 

Available 

0 

70 

Clean 

010118 

PHASE 

Triangular(60,63,67) 

78 

Gamma 

010119 

Available 

0 

21 

Beta 

010120 

Available 

0 

266 

ESB 

010121 

Available 

0 

238 

010122 

Available 

0 

91 

mam 

010123 

Available 

0 

Beta 

010124 

Available 

0 

mmummm  i 

Beta 

Table  3  describes  the  general  plan  for  the  schedule.  As  indicated  in  Table  3  the  number  of 
sorties  for  each  run  is  outlined  for  each  day  on  the  schedule.  In  column  2  for  Monday,  12/10 
means  that  the  schedule  will  require  that  we  fly  12  aircraft  in  the  first  run  and  turn  10  aircraft  for 
the  second  run. 


Table  3:  General  Schedule 


Sunday 

OFF 

Monday 

12/10 

Tuesday 

12/10 

Wednesday 

12/10 

Thursday 

12/10 

Friday 

10/8 

Saturday 

10/8 
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The  following  three  tables  describe  the  actual  schedule  the  aircraft  are  to  execute.  Table  4  will 
include  the  schedule  for  Monday  and  Tuesday.  Table  5  will  include  the  schedule  for  Wednesday 
and  Thursday.  Finally,  Table  6  will  include  the  schedule  for  Friday  and  Saturday.  Sunday  will 
not  be  on  the  schedule  due  to  the  fact  that  in  Table  3  it  was  defined  as  an  off  day.  In  the  first 
column  of  these  tables  are  the  tail  numbers  of  the  aircraft  followed  by  the  configuration.  The 
takeoff  and  landing  times  for  flight/run  one  are  then  defined  followed  by  the  takeoff  and  landing 
times  for  flight/run  2.  After  the  takeoff  and  landing  times  for  flight/run  2,  the  following  days 
schedule  begins  in  the  next  column. 


Table  4:  Sortie  Schedule  1 


Tail  # 

Mon. 

Config. 

FI 

TO 

Land 

F2 

TO 

Land 

Tucs. 

Conti". 

FI 

TO 

Land 

F2 

TO 

Land 

10101 

Clean 

0800 

0905 

1600 

1730 

Clean 

0800 

0905 

1600 

1730 

10102 

Clean 

0800 

0905 

1600 

1730 

Clean 

0800 

0905 

1600 

1730 

10103 

Clean 

0800 

0905 

1600 

1730 

Clean 

0800 

0905 

1600 

1730 

10104 

Clean 

SM 

Clean 

SPR 

10105 

Beta 

0750 

0915 

SPR 

Beta 

0750 

0915 

SPR 

10106 

Clean 

SPR 

Clean 

0800 

0905 

1600 

1730 

10107 

Gamma 

XCO 

Gamma 

XCO 

10108 

Gamma 

XCO 

Gamma 

XCO 

10109 

Elias 

0805 

0940 

SPR 

Alpha 

0805 

0940 

1605 

1720 

10110 

DEMO 

ESllEfl 

DEMO 

10111 

Beta 

0750 

0915 

1550 

1715 

Beta 

0750 

0915 

1550 

1715 

10112 

Alpha 

CANN 

mm 

CANN 

10113 

Clean 

SM 

Clean 

SM 

10114 

Clean 

DEMO 

Beta 

SPR 

10115 

EUaS 

NMC 

Alpha 

NMC 

10116 

Alpha 

SPR 

Alpha 

SPR 

10117 

Clean 

SPR 

Clean 

SPR 

10118 

Gamma 

PHASE 

Gamma 

PHASE 

10119 

Beta 

0750 

0915 

1550 

1715 

Beta 

0750 

0915 

1550 

1715 

10120 

Ellas 

0805 

0940 

1605 

1720 

0805 

0940 

SPR 

10121 

QiUK 

0805 

0940 

1605 

1720 

Elias 

0805 

0940 

1605 

1720 

10122 

mm 

0805 

0940 

1605 

1720 

Alpha 

0805 

0940 

1605 

1720 

10123 

Beta 

0750 

0915 

1550 

1715 

Beta 

SM 

10124 

Beta 

0750 

U915 

1550 

1715 

Beta 

0750 

0915 

1550 

1715 
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Table  5:  Sortie  Schedule  2 


1  Tail  # 

Wed. 

Config. 

FI 

TO 

Land 

F2 

TO 

Land 

Thur. 

Confi". 

FI 

TO 

Land 

F2 

TO 

Land 

10101 

Clean 

SM 

Clean 

SM 

10102 

Clean 

SPR 

Clean 

SPR 

10103 

Clean 

0800 

0905 

1600 

1730 

Clean 

PHASE 

10104 

Clean 

SPR 

Clean 

SPR 

10105 

Beta 

0750 

0915 

1550 

1715 

Beta 

0750 

0915 

1550 

1715 

10106 

Clean 

0800 

0905 

1600 

1730 

Clean 

0800 

0905 

1600 

1730 

10107 

Beta 

SPR 

Beta 

SPR 

10108 

Alpha 

0805 

0940 

1605 

1720 

Alpha 

0805 

0940 

1605 

1720 

10109 

ESia 

0805 

0940 

1605 

1720 

mm 

0805 

0940 

1605 

1720 

10110 

Alpha 

DEMO 

Alpha 

SM 

10111 

Beta 

0750 

0915 

1550 

1715 

Beta 

0750 

0915 

1550 

1715 

10112 

Alpha 

CANN 

Alpha 

CANN 

10113 

Clean 

SM 

Clean 

SM 

10114 

Beta 

SPR 

Beta 

SPR 

10115 

NMC 

Alpha 

0805 

0940 

1605 

1720 

10116 

IWWtti 

0805 

0940 

1605 

1720 

Alpha 

0805 

0940 

SPR 

10117 

Clean 

0800 

0905 

1600 

1730 

Clean 

0800 

0905 

1600 

1730 

10118 

Gamma 

PHASE 

Clean 

0800 

0905 

1600 

1730 

10119 

Beta 

0750 

0915 

SPR 

Beta 

0750 

0915 

SPR 

10120 

Alpha 

0805 

0940 

1605 

1720 

Alpha 

0805 

0940 

1605 

1720 

10121 

idiiiai 

SPR 

BIsSI 

SPR 

10122 

EJS3B 

0805 

0940 

SPR 

Alpha 

SPR 

10123 

Beta 

SM 

Beta 

0750 

0915 

1550 

1715 

10124 

Beta 

0750 

0915 

1550 

1715 

Beta 

SM 
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Table  6:  Sortie  Schedule  3 


Tail  # 

Fri. 

Config. 

FI 

TO 

Land 

F2 

TO 

Land 

Sat. 

Config. 

FJ 

TO 

Land 

F2 

TO 

Land 

10101 

Clean 

SM 

Clean 

SM 

10102 

Clean 

0800 

0905 

1600 

1730 

Clean 

0800 

0905 

1600 

1730 

10103 

Clean 

PHASE 

Clean 

PHASE 

10104 

Clean 

0800 

0905 

1600 

1730 

Clean 

SPR 

10105 

Beta 

XCO 

Beta 

XCO 

10106 

Clean 

SPR 

Clean 

SPR 

10107 

Beta 

SPR 

Beta 

SPR 

10108 

dm 

0805 

0940 

1605 

1720 

0805 

0940 

1605 

1720 

10109 

mm 

0805 

0940 

1605 

1720 

0805 

0940 

1605 

1720 

10110 

SM 

Alpha 

SM 

10111 

Beta 

XCO 

Beta 

XCO 

10112 

ESI  SB 

CANN 

CANN 

10113 

Clean 

SPR 

Clean 

0800 

0905 

1600 

1730 

10114 

Beta 

0750 

0915 

SPR 

Beta 

0750 

0915 

SPR 

10115 

ESI  IB 

0805 

0940 

1605 

1720 

m 

0805 

0940 

SPR 

10116 

Alpha 

0805 

0940 

SPR 

ES'diB 

0805 

0940 

1605 

1720 

10117 

Clean 

0800 

0905 

1600 

1730 

Clean 

SPR 

10118 

Clean 

SPR 

Clean 

0800 

0905 

1600 

1730 

10119 

Beta 

SPR 

Beta 

SPR 

10120 

ESHiZOM? 

0805 

0940 

1605 

1720 

mm 

SPR 

10121 

SPR 

mmm 

SPR 

10122 

ES3B 

SPR 

Alpha 

0805 

0940 

1605 

1720 

10123 

Beta 

0750 

0915 

1550 

1715 

Beta 

0750 

0915 

1550 

1715 

10124 

Beta 

SM 

Beta 

SM 

6.6  Simulation 

The  schedule  defined  above  along  with  initial  aircraft  states  was  used  as  input  to  the  simulation 
model.  The  simulation  was  set  to  run  for  five  replications  using  a  different  random  number 
stream  for  each  replication.  Each  replication  yielded  a  set  of  output  statistics.  Following  the 
initial  replications  we  used  the  simulation  to  perform  what-if  analysis.  The  following  sections 
will  describe  the  output  received  from  the  simulation  both  in  the  initial  replications  as  well  as  in 
the  “what  if’  analysis. 
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6. 6. 1  Output 

There  were  three  interesting  output  statistics  which  we  tracked  in  the  simulation  change 
opportunities,  phase  flow  line,  and  number  of  aircraft  down  in  need  of  repair.  Change 
opportunities  are  defined  as  number  of  sorties  not  executed  by  the  scheduled  aircraft.  This  gives 
an  indication  of  the  number  of  human  interactions  with  the  schedule  which  would  be  needed  to 
execute  the  schedule.  Phase  flow  line  describes  a  plot  of  the  hours  remaining  until  phase  across 
all  aircraft  sorted  in  ascending  order.  When  a  LRU  fails  on  an  aircraft,  that  aircraft  is  considered 
NMC  and  is  down  in  need  of  repair.  We  count  the  number  of  aircraft  down  needing  repair 
throughout  the  simulation.  Table  7  outlines  the  results  from  the  5  replications  of  the  schedule. 
The  results  provided  in  Table  7  are  averages  across  the  five  replications.  In  Table  7,  the  number 
of  sorties  for  each  day  is  displayed  along  with  the  number  of  sorties  completed  by  the  scheduled 
aircraft.  The  difference  between  these  two  values  is  the  number  of  change  opportunities. 
TCOWS  refers  to  the  Total  Change  Opportunities  for  the  Weekly  Schedule. 


Table  7:  Output  Data 


|l)ay 

Sorties  Scheduled  Completed  Sorties 

Changes  Needed 

Monday 

22 

21.4 

3SHHI 1 

3131 

22 

20 

2 

22 

18.6 

3.4 

22 

19.4 

2.6 

Friday 

18 

16.2 

1.8 

Saturday 

18 

16.4 

1.6 

Sunday 

WEKRHBKM 

Avg.  Total  Change  Opportunities  for  Weekly  Schedule 

12 

Standard  Deviation  of  Avg.  TCOWS 

3 

Average  Weekly  Total  #  of  Aircraft  Needing 
Unscheduled  Maintenance 

4 

Figure  20  details  the  phase  flow  before  the  schedule  was  executed  and  then  the  average  phase 
flow  across  the  five  replications  after  the  schedule  was  executed.  Also  indicated  on  this  graph  is 
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the  target  phase  flow  line.  The  target  slope  is  12.5  or  a  straight  line  from  0  to  300.  The  goal  of 
phase  flow  is  to  be  close  to  that  line  and  to  have  a  high  R2  value  meaning  that  all  values  fall  close 
to  the  line.  The  initial  phase  flow  was  defined  with  a  slope  of  9.84,  and  the  average  ending  slope 
was  9.64.  This  indicates  that  our  schedule  did  not  change  the  phase  flow  much  but  it  did  move  it 
slightly  away  from  target.  To  fix  the  phase  flow  in  the  graph  one  would  fly  more  hours  off  of  the 
aircraft  with  fewer  available  hours,  and  hold  back  the  aircraft  in  the  dip  in  the  middle  of  the 
graph.  This  would  increase  both  the  R2  and  the  slope  of  the  line. 


6.6.2  What-If Scenarios 

After  completing  our  initial  replications  of  the  schedule  we  performed  some  what-if  analysis. 
We  developed  two  scenarios  and  ran  the  simulation  with  slight  variations  in  input  parameters  to 
see  what  would  happen.  The  first  scenario  we  ran  was  one  where  supply  will  take  20%  longer  to 
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be  shipped  between  the  base  and  the  depot.  Table  8  outlines  the  summary  output  statistics  for 
this  scenario.  As  indicated  in  Table  8,  there  was  an  increase  in  the  number  of  change 
opportunities  as  well  as  the  variability  in  the  number  of  change  opportunities. 


Table  8:  Supply  What-If 


1  Da> 

Sorties  Scheduled 

Completed  Sorties 

Changes  Needed 

Monday1 

22 

21.4 

0.6 

1 

22 

20 

2 

22 

19 

3 

22 

18.2 

3.8 

Friday 

18 

15.2 

2.8 

18 

15.6 

2.4 

Sunday 

0 

■BBS 

Change  Opportunities 

14.6 

Standard  Deviation 

3.5777 

Average  Phase  Flow  Slope 

Average  Weekly  Total  #  of  Aircraft  Needing 
Unscheduled  Maintenance 

3.8 

The  second  scenario  involved  the  maintenance  and  installation  processes  taking  25%  longer  to 
complete  during  the  scheduled  week.  Table  9  outlines  the  summary  output  statistics  for  the 
second  scenario.  As  indicated  in  Table  9  increasing  the  maintenance  and  installation  times 
greatly  increased  the  number  of  change  opportunities  as  well  as  increasing  the  variability  in  the 
number  of  change  opportunities  experienced. 
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Table  9:  Maintenance  What-If 


Day 

Sorties  Scheduled  Completed  Sorties 

Changes  Needed 

22 

21.2 

0.8 

22 

19.6 

2.4 

22 

19.8 

2.2 

22 

17.6 

4.4 

he 

IS 

14.2 

3.8 

18 

14.6 

3.4 

Sunday 

0 

0 

Change  Opportunities 

17 

Standard  Deviation 

3.8730 

Average  Phase  Flow  Slope 

9.6272 

Average  Weekly  Total  #  of 
Unscheduled  Maintenance 

Aircraft  Needing 

4.8 

7  Summary  and  Conclusions 

In  the  course  of  this  report  we  have  introduced  the  concept  of  using  simulation  to  provide 
support  at  the  unit  level.  We  then  developed  a  simulation  model  based  on  an  USAF  sortie 
generation  problem.  This  simulation  was  used  to  execute  a  predefined  weekly  schedule  and 
analyze  its  effectiveness  also  allowing  the  user  to  perform  what-if  analysis  with  schedule  inputs. 
A  user  interface  was  also  developed  for  this  simulation  to  allow  users  to  quickly  and  easily 
provide  input  data  as  well  as  view  and  interpret  results.  These  two  pieces  when  placed  together 
constitute  the  goal  of  this  research  which  was  stated  as  the  development  of  a  proof  of  concept 
tool  demonstrating  the  usefulness  of  simulation  in  aiding  unit  level  decision  makers. 

One  of  the  major  limitations  of  the  system  was  that  the  data  used  were  from  theoretical 
distributions  were  not  from  a  real  world  sources.  The  Air  Force  has  vast  amounts  of  information 
available  about  the  wing  and  individual  weapon  systems.  Finding  a  way  to  directly  import  this 

information  would  be  extremely  valuable  to  a  system  such  as  the  one  described  in  this  report. 
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The  DSS  would  have  been  much  more  powerful  if  we  had  had  access  to  the  real  world 
information  and  to  the  data  formats  it  is  stored  in. 

Future  research  should  explore  the  possibility  of  adding  more  agents  to  the  decision  support 
system.  For  example,  in  our  research  we  used  a  simulation  model  to  perform  what-if  analysis  to 
help  validate  schedules.  An  additional  agent  could  be  created  that  searches  a  knowledge  base  for 
similar  scenarios  to  help  validate  the  schedule  against  past  schedules  under  similar 
circumstances.  Another  agent  might  optimize  the  schedule  against  a  certain  metric,  utilization 
for  instance,  and  then  the  schedule  could  be  run  in  what-if  scenario  for  further  analysis.  Further 
research  could  also  be  pursued  in  the  following  areas: 

•  Enhance  Simulation  Model 

-  Incorporate  more  change  logic  into  model 

-  Extend  model  to  other  aspects  of  flight  line 

-  Develop  and  capture  additional  performance  metrics 

•  Enhance  User  Interface 

-  Explore  portable  issues  so  it  can  be  used  on  the  flight  line  (i.e.  PDA,  web- 
interface) 

-  Allow  the  user  to  modify  the  simulation  so  it  more  closely  matches  their  specific 
situation 

-  Additional  visual  or  multi-criteria  displays  of  schedule  risk 

-  Additional  user  testing 

•  Examine  ways  to  generate  heuristic  or  optimal  schedules  to  the  user 

-  Model  schedule  as  optimization  problem 

•  Examine  deployment  issues 
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-  The  Air  Force  has  a  large  amount  of  data  available  in  multiple  formats. 

-  Connecting  decision  support  tools  such  as  the  one  developed  in  this  research  to 
those  data  sources  could  be  very  valuable 

-  Access  to  AF  data  formats  (i.e.  CAMS,  TASAMS) 

We  have  analyzed  the  challenges  associated  with  applying  simulation  based  decision 
support  systems  at  the  unit  level.  The  development  of  the  prototype  and  its  design/testing 
indicate  that  simulation  technology  augmented  by  enhanced  user  interaction  technology  clearly 
can  improve  the  decision  capabilities  of  unit  level  logistic  planners.  It  remains  to  further 
enhance  these  tools  with  the  additional  agent  concept,  and  to  develop  deployable  field  tests  to 
fully  evaluate  the  use  of  this  important  technology 
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Appendix  1 


Baseline  Model  Inputs  and  Distributions 


1  Parts 

Number  of  LRUs 

6 

Number  SRUs-LRUl 

4 

Number  SRUs-LRU2 

4 

Number  SRUs-LRU3 

4 

Number  SRUs-LRU4 

4 

Number  SRUs-LRU5 

4 

Number  SRUs-LRU6 

4 

Time  to  Failure  Distribution  for  SURs  in 

Exponential  (300) 

Hours  -  Each  of  these  distributions 

Exponential  (400) 

corresponds  to  a  set  of  eight  SRUs  within  the 
system. 

Exponential  (500) 

Inventory  at  Base  Level  for  all  SRUs 

3 

Inventory  at  Depot  Level  for  all  SRUs 

20 

Repair  Time  Distribution  for  all  SRUs  in 

Hours 

Exponential  (8) 

Number  of  Repair  Resources  at  the  Base 

Level 

Unlimited  Capacity 

Number  of  Repair  Resources  at  the  Depot 

Level 

Unlimited  Capacity 

Queuing  rules  for  repair 

Backorders  parts  are  given  top 
priority  in  the  repair  process 

Probability  that  a  Part  can  be  Repaired  at  the 
Base  Level 

.01 

Shipping 

_ 
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Numbc  Distribution  (Hours)  Description 

i' 

1 

TRIA(12,81.6,184.8) 

This  distribution  is  used  to  generate 
shipping  times  between  bases  in  the 
same  region. 

2 

TRIA(3 1.2, 170.4, 348) 

This  distribution  is  used  to  generate 
shipping  times  between  the  depot 
and  the  bases  in  region  1. 

3 

TRIA(76.8, 266.4, 

453.6) 

This  distribution  is  used  to  generate 
shipping  times  between  the  depot 
and  the  bases  in  region  2.  This 
distribution  is  also  used  to  generate 
the  shipping  time  between  bases 
which  are  not  in  the  same  region. 

Shipping  Time  Distribution  Matrix 


Basel 


Base2 


Base3 


Base4 


BaseS 


Basel 


0 


1 


Base2 


1 


0 


1 


Base4 


3 


Jana 


MICAP  Shipping  Time  in  Hours 


Maximum  LTL  Load  in  Number  of  SRUs  20 


Minimum  LTL  Load  in  Percent  of  Max  Load  0.2 


Structure  (refer  to  Figure 


Base6 


3 


3 


3 


1 


1 


Number  of  Planes  Per  Unit 

24 

Number  of  Units  Per  Squadron 

3 

Number  of  Squadrons  Per  Base 

1 

Number  of  Bases 

6 

Number  of  Regions 

2 

Number  of  Bases  Per  Region 

3 

Number  of  Depots 

1 

Phase  Inspection 


Accrued  Flight  Hours  Before  Base  phase 
inspection 

290-305  Flight  Hours 

Base  Phase  Inspection  Time  Distribution  in 
Days 

Triangular  (5,7,10) 

Repair  Resources  Required  for  Phase 

Inspection 

50 

Queuing  Rules  for  Phase  Inspection 

First  In  First  Out 

Pre  Flight  Operations 


Refuel 

Triangular  (8,10,12)  minutes 

Weapon  Load 

Triangular(25,30,35)  minutes 

Pre-Flight 

Triangular(50,60,70)  minutes 

Engine  Start,  Final  Systems  Check,  and 

Taxiing 

Triangular(7,10,12)  minutes 

Takeoff 

Triangular(2,3,4)  minutes 

Post  Flight  Operations  | 

Landing 

Triangular(14,15,16)  minutes 

Parking  and  Recovery 

Triangular(5,7,9)-2  minutes 

Final  Parking 

2  Minutes  with  LRU  clock  stopped 

Service  and  Debrief 

Triangular(45,60,75)  minutes 

Unscheduled  Maintenance  Operations  1 

Troubleshooting 

Triangular(20,24,30)  minutes 

Remove  Part 

Triangular(45,60,70)  minutes 

Wait  for  Part  to  issue  from  supply 

Triangular(.5, 2,2.5)  hours 

Delay  for  paperwork 

Triangular(5,10,15)  minutes 

Installation 

Triangular(60,84,120)  minutes 

Operational  Check 

Triangular(l  5,20,25)  minutes 

Signoff  Discrepancy 

Triangular(5,10,15)  minutes 

Document  Corrective  Action 

Triangular(5,10,15)  minutes 
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